-- Logs begin at Tue 2022-09-13 17:19:58 UTC, end at Tue 2022-09-13 18:21:14 UTC. -- Sep 13 17:46:06.315449 dv-cinder-852009-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 17:46:06.323290 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 17:46:06 2022] *** Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: nodename: dv-cinder-852009-8-iscsi Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: machine: x86_64 Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: clock source: unix Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: pcre jit disabled Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: detected number of CPU cores: 4 Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: current working directory: / Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: detected binary path: /usr/bin/uwsgi-core Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: your processes number limit is 47680 Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: your memory page size is 4096 bytes Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: detected max file descriptor number: 2048 Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: lock engine: pthread robust mutexes Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: thunder lock: enabled Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 17:46:06.327169 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 17:46:06.365832 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: Python main interpreter initialized at 0x55fc678149c0 Sep 13 17:46:06.365832 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: python threads support enabled Sep 13 17:46:06.365832 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: your server socket listen backlog is limited to 100 connections Sep 13 17:46:06.365832 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: your mercy for graceful operations on workers is 90 seconds Sep 13 17:46:06.366262 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 17:46:06.366262 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: *** Operational MODE: preforking *** Sep 13 17:46:06.366527 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: *** uWSGI is running in multiple interpreter mode *** Sep 13 17:46:06.366527 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: spawned uWSGI master process (pid: 81416) Sep 13 17:46:06.366773 dv-cinder-852009-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 17:46:06.368314 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: spawned uWSGI worker 1 (pid: 81428, cores: 1) Sep 13 17:46:06.368314 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: spawned uWSGI worker 2 (pid: 81429, cores: 1) Sep 13 17:46:06.368314 dv-cinder-852009-8-iscsi devstack@keystone.service[81416]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 17:46:10.607884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 17:46:10.607884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: from cryptography.utils import int_from_bytes Sep 13 17:46:10.608759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 17:46:10.608759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: from cryptography.utils import int_from_bytes Sep 13 17:46:10.609177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 17:46:10.609177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: from cryptography.utils import int_from_bytes Sep 13 17:46:10.609286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 17:46:10.609286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: from cryptography.utils import int_from_bytes Sep 13 17:46:10.658169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 17:46:10.658370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 17:46:10.658471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 17:46:10.658553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] command line args: [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 17:46:10.658619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 17:46:10.658765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] command line args: [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 17:46:10.658907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 17:46:10.658907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ================================================================================ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 17:46:10.659077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 17:46:10.659155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] admin_token = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.659233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ================================================================================ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 17:46:10.659447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_dir = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.659554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] admin_token = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.659626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.659692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_dir = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.659756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] config_source = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.659819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.659910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] debug = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_source = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] debug = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.660865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_config_append = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_config_append = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_dir = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.661977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_dir = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.662982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.663919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.664936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.665016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.665141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.665225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.665305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.665434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.665610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.665873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] publish_errors = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] publish_errors = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.666887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] standard_threads = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] standard_threads = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.667980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.668198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.668318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_journal = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.668456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.668635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_json = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.668760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_journal = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.668880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_stderr = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.668995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_json = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.669114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] use_syslog = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.669235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_stderr = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.669363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.669479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_syslog = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.669479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 17:46:10.669691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.669691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.669916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.669916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670590 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.670944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.671985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.672945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.673997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.674195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.674319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.674439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.674553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.674672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.674787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.674912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.675951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676176 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676835 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.676950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.677072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.677187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.677299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.677418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.677548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.678911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.679974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.680919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.681794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.682920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.683977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.684967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685424 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.685888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.686948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.687886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.688936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.689977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.690892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.691915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.692897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.693941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.694921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.external = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.external = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.695942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.password = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.password = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] auth.token = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.token = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.696924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.697978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.698916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.699985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.700989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.701945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.702129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.702256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.702355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.702433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.702610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.702776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.702905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.703961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.704972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.705907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.706965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.707908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708261 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.708973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.709897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.710929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.711987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.712979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.713933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.714971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.715994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.716918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.717958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.718897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.719933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.720927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721507 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.721970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.722914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723261 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.723963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.724910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.725966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.726966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.727089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.727207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.727321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.727436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.727550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.727665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.727788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.728331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.728492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.728599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.728699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.728798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.728911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.729910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.730918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.731996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.732978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.733911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.734964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.735888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.736989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.737918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.738888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.739939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.740897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741828 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.741943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.742058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.742202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.742349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.742480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.742480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.742743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.743025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.743183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81429) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 17:46:10.743313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.743447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.743693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 17:46:10.743867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 17:46:10.772848 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.773053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.773280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.773389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.773670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.773899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.781907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.782870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.813578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.813922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.818607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.818840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.870103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.871116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.918158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.918339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.918823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:10.919060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.079998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.080211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.080462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.080678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.080867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.081077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.088442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.088709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.088914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.089150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.089363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.089568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.104909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.105062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.105150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.105275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.105357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.105500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.105740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.116516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.116656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.129338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.129483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.129628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.129736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.129865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.129945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.130588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81429) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 17:46:11.147635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.147817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.147952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 17:46:11.148090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 17:46:11.148184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.148353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.148472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 17:46:11.148592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 17:46:11.148673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.148858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.149005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.149144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.149225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.149366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.149493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.149631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.159374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.159503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.159633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.159792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.159918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.160070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.160158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.160319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.167192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.167322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.170100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.170274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.170403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.170520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.170618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.170696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.170923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.171943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.172061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.172233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.172327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.172545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.172669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.172795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.172886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.195239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.195497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.196865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.197022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.197157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.197316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.201808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.201939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.202059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.202144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.202355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.202482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.208628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.208786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.208929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.209055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.214842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.214996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.215122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.215327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.215455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.215601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.215690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.215867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.215995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.216134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.216230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.216406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.216557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.216695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.216777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.216932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.232199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.232510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.232637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.232780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.236346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.236637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.239538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.239831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.239958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.240091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.244042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.244352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.247155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.247501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.250423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.250717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.250843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.250983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.251069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.251264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.251411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.251498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.251576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.251821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.251950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.252104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.268025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.268312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.268462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.268599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.268684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.268847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.268975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.269158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.275784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.276040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.277350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.277627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.278867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.279139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.279345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.279438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.281801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.282080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.282207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.282416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.282556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.282655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.282734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.282974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.283118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.283371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.294189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.294477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.294603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.294762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.294891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.295007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.295170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.295399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.295527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.295688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.295830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.295914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.296096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.296380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.311299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.311995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.314241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.314991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.317131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.317932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.318548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.319384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.320392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.320700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.321045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.321173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.321356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.321740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.327337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.328096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.328804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.329598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.331642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.332025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.332311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.332516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.332707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.332892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.333185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.333465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.341449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.341753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.342041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.342527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.342835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.343117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.348979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.349269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.349555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.349842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.350020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81429) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 17:46:11.350107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.350210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.350373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.350488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.350648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.350802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.350928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.351047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.351211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.351332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.351508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.351612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.351778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.351864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.352041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.352128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.352307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.352394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.352896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.352979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.352979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.353166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.353424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81429) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 17:46:11.386920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fc678149c0 pid: 81428 (default app) Sep 13 17:46:11.387114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fc678149c0 pid: 81429 (default app) Sep 13 17:46:11.390777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a484a-35bd-4c1d-a838-4b6adff28ccd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:11.391045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a484a-35bd-4c1d-a838-4b6adff28ccd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:11.391348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a484a-35bd-4c1d-a838-4b6adff28ccd None None] PATH_INFO: `/v3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:11.395598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Tue Sep 13 17:46:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:18.293700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f032889-7a96-4219-bd4f-61babd0badb6 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:18.294253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f032889-7a96-4219-bd4f-61babd0badb6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:18.294253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f032889-7a96-4219-bd4f-61babd0badb6 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:18.298594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:18.304862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:18.305345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:18.305345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:18.382167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: INFO dbcounter [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] Registered counter for database keystone Sep 13 17:46:18.402281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG oslo_db.sqlalchemy.engines [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 17:46:18.409838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writer thread running {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 17:46:18.590106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG passlib.handlers.bcrypt [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81428) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 17:46:18.590384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG passlib.handlers.bcrypt [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81428) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 17:46:18.650255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:18.651873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:18.794837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 17:46:19.259498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f032889-7a96-4219-bd4f-61babd0badb6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:19.329427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: INFO dbcounter [None req-6f032889-7a96-4219-bd4f-61babd0badb6 None None] Registered counter for database keystone Sep 13 17:46:19.352366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG oslo_db.sqlalchemy.engines [None req-6f032889-7a96-4219-bd4f-61babd0badb6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81429) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 17:46:19.359875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writer thread running {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 17:46:19.482006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f032889-7a96-4219-bd4f-61babd0badb6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:19.489082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfcb48-3dd2-41a7-9f7a-a6bdc634ec3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:19.490106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfcb48-3dd2-41a7-9f7a-a6bdc634ec3f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:19.490566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfcb48-3dd2-41a7-9f7a-a6bdc634ec3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:19.491019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfcb48-3dd2-41a7-9f7a-a6bdc634ec3f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:19.506074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfcb48-3dd2-41a7-9f7a-a6bdc634ec3f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:19.558845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING keystone.common.rbac_enforcer.enforcer [None req-22dfcb48-3dd2-41a7-9f7a-a6bdc634ec3f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 17:46:19.559194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfcb48-3dd2-41a7-9f7a-a6bdc634ec3f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:19.569702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Tue Sep 13 17:46:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 17:46:19.576082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:19.597964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036ba7c-9401-4270-bd46-73a3e44b3184 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:19.603531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba940f-39b2-4387-b204-19bd1f4954f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:19.604727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba940f-39b2-4387-b204-19bd1f4954f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:19.605007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba940f-39b2-4387-b204-19bd1f4954f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:19.605279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba940f-39b2-4387-b204-19bd1f4954f1 admin admin] PATH_INFO: `/v3/projects/c326683f56fe42d09b126fc368517cb3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:19.606975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba940f-39b2-4387-b204-19bd1f4954f1 admin admin] RBAC: Authorizing `identity:get_project(project_id=c326683f56fe42d09b126fc368517cb3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:19.659887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e4ba940f-39b2-4387-b204-19bd1f4954f1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 17:46:19.660233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba940f-39b2-4387-b204-19bd1f4954f1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:19.665070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Tue Sep 13 17:46:19 2022] GET /identity/v3/projects/c326683f56fe42d09b126fc368517cb3 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:21.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a74ad8-19ac-407c-8ea8-4b262c4c1611 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:21.774313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a74ad8-19ac-407c-8ea8-4b262c4c1611 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:21.774313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a74ad8-19ac-407c-8ea8-4b262c4c1611 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:21.778528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:21.784225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1ff9b-c544-41c8-add4-8cbe9892ba24 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:21.784705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1ff9b-c544-41c8-add4-8cbe9892ba24 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:21.784796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1ff9b-c544-41c8-add4-8cbe9892ba24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:21.806137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8ea1ff9b-c544-41c8-add4-8cbe9892ba24 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:21.807722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8ea1ff9b-c544-41c8-add4-8cbe9892ba24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:21.815955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 17:46:22.078481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a74ad8-19ac-407c-8ea8-4b262c4c1611 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:22.092752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a74ad8-19ac-407c-8ea8-4b262c4c1611 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:22.097581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ae16-716e-453d-bc8c-60c2917f0572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:22.098216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ae16-716e-453d-bc8c-60c2917f0572 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:22.098493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ae16-716e-453d-bc8c-60c2917f0572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:22.098751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ae16-716e-453d-bc8c-60c2917f0572 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:22.106592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ae16-716e-453d-bc8c-60c2917f0572 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:22.108389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ae16-716e-453d-bc8c-60c2917f0572 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:22.117184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Tue Sep 13 17:46:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 17:46:22.122982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1ff9b-c544-41c8-add4-8cbe9892ba24 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:22.132825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1ff9b-c544-41c8-add4-8cbe9892ba24 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:22.138988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94255965-3bd9-4880-9881-faaf1792b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:22.139621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94255965-3bd9-4880-9881-faaf1792b9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:22.139888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94255965-3bd9-4880-9881-faaf1792b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:22.140153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94255965-3bd9-4880-9881-faaf1792b9e2 admin admin] PATH_INFO: `/v3/users/bb561c12b9c44bd29f9bae620dc33608` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:22.141644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94255965-3bd9-4880-9881-faaf1792b9e2 admin admin] RBAC: Authorizing `identity:get_user(user_id=bb561c12b9c44bd29f9bae620dc33608)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:22.143518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94255965-3bd9-4880-9881-faaf1792b9e2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:22.145129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Tue Sep 13 17:46:22 2022] GET /identity/v3/users/bb561c12b9c44bd29f9bae620dc33608 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:24.355073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045ee56-c8f7-4aff-bc3b-20e913b6342d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:24.355623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045ee56-c8f7-4aff-bc3b-20e913b6342d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:24.355623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045ee56-c8f7-4aff-bc3b-20e913b6342d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:24.357061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:24.362400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaff672-889f-41d7-8120-3f8957997354 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:24.362866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaff672-889f-41d7-8120-3f8957997354 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:24.362961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaff672-889f-41d7-8120-3f8957997354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:24.382201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-afaff672-889f-41d7-8120-3f8957997354 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:24.383475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-afaff672-889f-41d7-8120-3f8957997354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:24.401586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d7af-cc01-4a55-8a23-8af3f6f695c8 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:24.401880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d7af-cc01-4a55-8a23-8af3f6f695c8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:24.402150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d7af-cc01-4a55-8a23-8af3f6f695c8 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:24.403504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 6/11] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:46:24.408896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:24.409191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:24.409450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:24.452082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG passlib.handlers.bcrypt [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81429) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 17:46:24.452492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG passlib.handlers.bcrypt [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81429) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 17:46:24.468655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 6/12] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:24.501290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:24.502921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:24.608301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 7/13] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:24.730901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaff672-889f-41d7-8120-3f8957997354 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:24.756218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:24.840943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaff672-889f-41d7-8120-3f8957997354 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:24.876968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:24.877537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:24.877772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:24.877880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:24.878075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:24.890440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:24.900874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:24.901895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:24.902720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 7/14] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:46:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:24.908656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:24.994861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a3d4a-05c6-41e8-848d-3ec5fd183561 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:24.997438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:24.999223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:25.017625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7a9c-fb66-441d-9cf9-a6c7a37ff502 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:25.036383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:25.037130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:25.037413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:25.037680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:25.053377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:25.054024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:25.054440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:25.054636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:25.055354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:25.055649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:25.057108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:25.069605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 8/15] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:25.075558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:25.124034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:25.125649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:25.181200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245af9d8-6fef-4294-963f-03456380d757 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:25.217718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:25.218457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:25.218735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:25.219008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] PATH_INFO: `/v3/users/bb561c12b9c44bd29f9bae620dc33608` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:25.220170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] RBAC: Authorizing `identity:get_user(user_id=bb561c12b9c44bd29f9bae620dc33608)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:25.221685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:25.227857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 9/16] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:25 2022] GET /identity/v3/users/bb561c12b9c44bd29f9bae620dc33608 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:25.228855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 8/17] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 17:46:25.228960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:25.234485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:25.238188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:25.247904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:25.335607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e76eb-1f37-40a9-ac35-863c142cc46d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:25.338445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae99cb5-3204-4f75-9505-751f3fd68719 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:25.371524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:25.372180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:25.372570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:25.372712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:25.372850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:25.372987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:25.373129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:25.373399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:25.374625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:25.376103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:25.377704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 10/18] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:25.383818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:25.392595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:25.394094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:46:25.412759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:25.413786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:46:25.414769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:46:25.414769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:46:25.414769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:25.414769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 9/19] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:46:25.419923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:25.469467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:25.489357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170a044-a16d-45d4-952f-1b8009d43654 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:25.504077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:25.525906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1f7a2-ade3-48b3-aa80-702db8cc0d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:25.526532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1f7a2-ade3-48b3-aa80-702db8cc0d5a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:25.526815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1f7a2-ade3-48b3-aa80-702db8cc0d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:25.527089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1f7a2-ade3-48b3-aa80-702db8cc0d5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:25.527835 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1f7a2-ade3-48b3-aa80-702db8cc0d5a None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.domain.id=default, user.id=bb561c12b9c44bd29f9bae620dc33608) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:25.528146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1f7a2-ade3-48b3-aa80-702db8cc0d5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:25.529662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1f7a2-ade3-48b3-aa80-702db8cc0d5a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:25.537142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 11/20] 10.222.0.43 () {56 vars in 1322 bytes} [Tue Sep 13 17:46:25 2022] GET /identity/v3/role_assignments?user.id=bb561c12b9c44bd29f9bae620dc33608&scope.domain.id=default&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:25.586688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b2040-4d54-4c16-b82c-9a2b1b9ce9b2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:25.631551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea967b-13bc-4e87-bfe2-e9289a5c6a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:25.632183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea967b-13bc-4e87-bfe2-e9289a5c6a36 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:25.632552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea967b-13bc-4e87-bfe2-e9289a5c6a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:25.632851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea967b-13bc-4e87-bfe2-e9289a5c6a36 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:25.633538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea967b-13bc-4e87-bfe2-e9289a5c6a36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:25.633841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea967b-13bc-4e87-bfe2-e9289a5c6a36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:25.635319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea967b-13bc-4e87-bfe2-e9289a5c6a36 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:25.656649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 10/21] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:27.845077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c70a0-9383-43d7-a64d-eec181a0e414 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:27.845601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c70a0-9383-43d7-a64d-eec181a0e414 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:27.845601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c70a0-9383-43d7-a64d-eec181a0e414 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:27.846993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 12/22] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:27.853214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de247dad-12e1-42f6-8089-b9e9cd728e47 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:27.853759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de247dad-12e1-42f6-8089-b9e9cd728e47 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:27.853980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de247dad-12e1-42f6-8089-b9e9cd728e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:27.882714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-de247dad-12e1-42f6-8089-b9e9cd728e47 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:27.883989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-de247dad-12e1-42f6-8089-b9e9cd728e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:27.986278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 11/23] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 17:46:28.368573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af4b05-e5fd-48af-b80c-b1ee0cf61810 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:28.368919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af4b05-e5fd-48af-b80c-b1ee0cf61810 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:28.369129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af4b05-e5fd-48af-b80c-b1ee0cf61810 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:28.370870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 13/24] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:28.376939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef4c2c-a081-4d6b-bd6b-797a96e6a158 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:28.377230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef4c2c-a081-4d6b-bd6b-797a96e6a158 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:28.377510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef4c2c-a081-4d6b-bd6b-797a96e6a158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:28.413511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d7ef4c2c-a081-4d6b-bd6b-797a96e6a158 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:28.414797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d7ef4c2c-a081-4d6b-bd6b-797a96e6a158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:28.479728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:28.514808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 12/25] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:28.542114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af4b05-e5fd-48af-b80c-b1ee0cf61810 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:28.684898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af4b05-e5fd-48af-b80c-b1ee0cf61810 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:28.737782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:28.738753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:28.739254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:28.739759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:28.746788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a31f1-215d-4a74-beff-0e635e072bad None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:28.747081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a31f1-215d-4a74-beff-0e635e072bad None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:28.747355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a31f1-215d-4a74-beff-0e635e072bad None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:28.748761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 13/26] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:28.760533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:28.764174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1acb8-0af3-434a-8378-d8c16099d186 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:28.764495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1acb8-0af3-434a-8378-d8c16099d186 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:28.764778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1acb8-0af3-434a-8378-d8c16099d186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:28.767761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:28.768604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:28.769364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:28.771631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 14/27] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:46:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:46:28.777741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:28.792928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:28.805797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-20c1acb8-0af3-434a-8378-d8c16099d186 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:28.807212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-20c1acb8-0af3-434a-8378-d8c16099d186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:28.968110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 14/28] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:28.973474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1acb8-0af3-434a-8378-d8c16099d186 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:29.009794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2287edb-c59a-4306-b20e-4563618de530 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:29.117941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:29.118657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:29.118947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:29.119220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:29.119911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:29.120213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:29.121666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:29.135852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1acb8-0af3-434a-8378-d8c16099d186 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:29.145890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 15/29] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:29.146822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:29.153617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:29.194090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:29.194813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:29.195153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:29.195395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:29.212505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:29.214030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:46:29.238177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:29.240303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:46:29.242344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:46:29.242344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:46:29.242344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:29.242344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 15/30] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:29.249515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:29.339917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:29.431619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0428e-39d8-4b27-b871-c55990b314f1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:29.485780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:29.486457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:29.486740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:29.487019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] PATH_INFO: `/v3/users/bb561c12b9c44bd29f9bae620dc33608` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:29.493407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] RBAC: Authorizing `identity:get_user(user_id=bb561c12b9c44bd29f9bae620dc33608)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:29.494865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:29.496884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 16/31] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:29 2022] GET /identity/v3/users/bb561c12b9c44bd29f9bae620dc33608 => generated 247 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:29.497407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:29.521156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739136b6-7f49-4e68-9697-03e5570fb50b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:29.577453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:29.578449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:29.578915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:29.579383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:29.581063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:29.581851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:29.583627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:29.604124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 16/32] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:29.605480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:29.635034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6472b1-a1c0-4ad8-af95-aa1c03985528 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:29.635439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:29.686586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:29.686901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:29.687242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:29.687384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:29.688087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:29.689607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:29.694367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:29.781010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116098b-f01b-41cf-ae13-a73581cca303 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:29.847415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c0d7-77fd-4961-927c-166ce7913e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:29.849216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c0d7-77fd-4961-927c-166ce7913e31 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:29.850044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c0d7-77fd-4961-927c-166ce7913e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:29.850749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c0d7-77fd-4961-927c-166ce7913e31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:29.853554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c0d7-77fd-4961-927c-166ce7913e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:29.856731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c0d7-77fd-4961-927c-166ce7913e31 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:29.860419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 17/33] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:29.863171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531e8e7-a245-4f11-a883-208860ec2b50 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:29.863969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531e8e7-a245-4f11-a883-208860ec2b50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:29.864811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531e8e7-a245-4f11-a883-208860ec2b50 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:29.865165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 17/34] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 13 17:46:29 2022] POST /identity/v3/roles => generated 219 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:29.866167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:29.874976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 18/35] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:29 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:29.877201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531e8e7-a245-4f11-a883-208860ec2b50 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:30.077929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d0f20-a5cc-4b59-83c6-891945ffa526 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:30.082829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:30.119968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206cb4-4e7c-4c52-8f48-eb99f7e08afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:30.120649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206cb4-4e7c-4c52-8f48-eb99f7e08afc None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:30.120998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206cb4-4e7c-4c52-8f48-eb99f7e08afc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:30.121275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206cb4-4e7c-4c52-8f48-eb99f7e08afc None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:30.122021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206cb4-4e7c-4c52-8f48-eb99f7e08afc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:30.123767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206cb4-4e7c-4c52-8f48-eb99f7e08afc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:30.124226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531e8e7-a245-4f11-a883-208860ec2b50 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:30.168743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 18/36] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:46:29 2022] POST /identity/v3/projects => generated 301 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:30.172841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0fb8-8862-4a6d-b027-2df53a8a69d0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:30.173857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0fb8-8862-4a6d-b027-2df53a8a69d0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:30.174702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0fb8-8862-4a6d-b027-2df53a8a69d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:30.216630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e508-0cea-43e5-a847-0ff0c573f408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:30.217387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e508-0cea-43e5-a847-0ff0c573f408 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:30.217670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e508-0cea-43e5-a847-0ff0c573f408 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:30.217980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e508-0cea-43e5-a847-0ff0c573f408 None admin] PATH_INFO: `/v3/domains/default/users/bb561c12b9c44bd29f9bae620dc33608/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:30.221403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e508-0cea-43e5-a847-0ff0c573f408 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bb561c12b9c44bd29f9bae620dc33608, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:30.221805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-06cb0fb8-8862-4a6d-b027-2df53a8a69d0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:30.225014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e508-0cea-43e5-a847-0ff0c573f408 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:30.225488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-06cb0fb8-8862-4a6d-b027-2df53a8a69d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:30.319806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:30.360506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 19/37] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:30.361562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60807c3b-951d-49ca-b1c5-58b565fad248 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:30.361865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60807c3b-951d-49ca-b1c5-58b565fad248 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:30.362120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60807c3b-951d-49ca-b1c5-58b565fad248 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:30.364856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 20/38] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:30.371646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849408a2-f8ef-4be4-94f6-78c768fdb019 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:30.372076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849408a2-f8ef-4be4-94f6-78c768fdb019 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:30.372501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849408a2-f8ef-4be4-94f6-78c768fdb019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:30.406508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 19/39] 10.222.0.43 () {58 vars in 1361 bytes} [Tue Sep 13 17:46:29 2022] PUT /identity/v3/domains/default/users/bb561c12b9c44bd29f9bae620dc33608/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:46:30.413833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-849408a2-f8ef-4be4-94f6-78c768fdb019 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:30.415338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-849408a2-f8ef-4be4-94f6-78c768fdb019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:30.488300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f589b-3eb1-4cef-9d94-f947795d88c3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:30.488626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f589b-3eb1-4cef-9d94-f947795d88c3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:30.488945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f589b-3eb1-4cef-9d94-f947795d88c3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:30.493926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 20/40] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:46:30.499953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dbae7-2ca8-4740-b2f0-cc737dc0818d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:30.500345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dbae7-2ca8-4740-b2f0-cc737dc0818d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:30.500560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49dbae7-2ca8-4740-b2f0-cc737dc0818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:30.529558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e49dbae7-2ca8-4740-b2f0-cc737dc0818d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:30.531356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e49dbae7-2ca8-4740-b2f0-cc737dc0818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:30.544002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 21/41] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:30.631331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 21/42] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:30.746203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849408a2-f8ef-4be4-94f6-78c768fdb019 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:30.852145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dbae7-2ca8-4740-b2f0-cc737dc0818d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:30.870486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849408a2-f8ef-4be4-94f6-78c768fdb019 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:30.944456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:30.970825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49dbae7-2ca8-4740-b2f0-cc737dc0818d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:30.993518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:31.008358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:31.009119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:31.009467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:31.009807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:31.010567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:31.012202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:31.028510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:31.029436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:31.029891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:31.030321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:31.031246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:31.032961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:31.037162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 22/43] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 13 17:46:30 2022] POST /identity/v3/roles => generated 223 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:46:31.052087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 22/44] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 13 17:46:30 2022] POST /identity/v3/roles => generated 225 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:31.147060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:31.315773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb82dd-5078-432b-a0bf-0d83d9795cb9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:31.358290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b64a32-1206-42f1-bb71-9d40efe3f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:31.358955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b64a32-1206-42f1-bb71-9d40efe3f627 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:31.359242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b64a32-1206-42f1-bb71-9d40efe3f627 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:31.359516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b64a32-1206-42f1-bb71-9d40efe3f627 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:31.360743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b64a32-1206-42f1-bb71-9d40efe3f627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:31.362242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b64a32-1206-42f1-bb71-9d40efe3f627 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:31.364111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 23/45] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:31.369862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:31.476329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b2fa5-7659-4f62-8878-b826dc30cb1c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:31.514934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fb0a3-7766-4cd7-8003-1279f84e170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:31.515571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fb0a3-7766-4cd7-8003-1279f84e170d None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:31.515849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fb0a3-7766-4cd7-8003-1279f84e170d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:31.516115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fb0a3-7766-4cd7-8003-1279f84e170d None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:31.516862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fb0a3-7766-4cd7-8003-1279f84e170d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:31.518401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fb0a3-7766-4cd7-8003-1279f84e170d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:31.548412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 23/46] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:46:31 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:46:32.981078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ddb7b-5bf5-4d2c-803c-17075bf1772a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:32.981912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ddb7b-5bf5-4d2c-803c-17075bf1772a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:32.981912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ddb7b-5bf5-4d2c-803c-17075bf1772a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:32.983054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 24/47] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:46:32.989114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603981f-0831-472a-bba7-ed80062d35cd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:32.989740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603981f-0831-472a-bba7-ed80062d35cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:32.989740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603981f-0831-472a-bba7-ed80062d35cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:33.009804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3603981f-0831-472a-bba7-ed80062d35cd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:33.011159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3603981f-0831-472a-bba7-ed80062d35cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:33.029972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:33.123341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 24/48] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:33.388626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ddb7b-5bf5-4d2c-803c-17075bf1772a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:33.393095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:33.503289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ddb7b-5bf5-4d2c-803c-17075bf1772a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:33.542645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:33.543308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:33.543591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:33.543868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:33.557857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:33.564172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:33.565128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:33.565889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 25/49] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:33.573122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603981f-0831-472a-bba7-ed80062d35cd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:33.692123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603981f-0831-472a-bba7-ed80062d35cd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:33.729914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531b078-7e92-49b7-85e0-b51031942f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:33.730552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531b078-7e92-49b7-85e0-b51031942f5d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:33.730866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531b078-7e92-49b7-85e0-b51031942f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:33.731109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531b078-7e92-49b7-85e0-b51031942f5d None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:33.731800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531b078-7e92-49b7-85e0-b51031942f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:33.732102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531b078-7e92-49b7-85e0-b51031942f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:33.733593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531b078-7e92-49b7-85e0-b51031942f5d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:33.742758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 25/50] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:33.748896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:33.788656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db9df-b155-4422-ba8f-bfebbf58c6fb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:33.788902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db9df-b155-4422-ba8f-bfebbf58c6fb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:33.789198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db9df-b155-4422-ba8f-bfebbf58c6fb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:33.791029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 26/51] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:33.796491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8367a10-bcd6-4a2c-b29c-520a13447cb4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:33.796811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8367a10-bcd6-4a2c-b29c-520a13447cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:33.797087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8367a10-bcd6-4a2c-b29c-520a13447cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:33.813633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:33.817045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e8367a10-bcd6-4a2c-b29c-520a13447cb4 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:33.818275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e8367a10-bcd6-4a2c-b29c-520a13447cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:33.822332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:33.856184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d0ab2-0ebb-4b28-ac12-9d77bb9a9d8d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:33.889562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 27/52] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:33.892207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:33.892891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:33.893172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:33.893446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] PATH_INFO: `/v3/users/bb561c12b9c44bd29f9bae620dc33608` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:33.894627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] RBAC: Authorizing `identity:get_user(user_id=bb561c12b9c44bd29f9bae620dc33608)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:33.896079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:33.897706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 26/53] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:33 2022] GET /identity/v3/users/bb561c12b9c44bd29f9bae620dc33608 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:33.903395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8367a10-bcd6-4a2c-b29c-520a13447cb4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:34.008602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8367a10-bcd6-4a2c-b29c-520a13447cb4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:34.044548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:34.045196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:34.045509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:34.045792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:34.047189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:34.048979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:34.050763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 28/54] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:34.056830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:34.152748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:34.162989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0c0ce-d6fa-4fd5-b4d7-56d07a1d431b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:34.165239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:34.194491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:34.203888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:34.205088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:34.205426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:34.205924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:34.207277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.domain.id=default, user.id=bb561c12b9c44bd29f9bae620dc33608) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:34.208134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:34.209753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:34.218587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 27/55] 10.222.0.43 () {56 vars in 1322 bytes} [Tue Sep 13 17:46:34 2022] GET /identity/v3/role_assignments?user.id=bb561c12b9c44bd29f9bae620dc33608&scope.domain.id=default&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:34.273970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f359c-58c8-4ea9-911b-d75aa85dce18 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:34.314185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4ee7e-9e49-4f49-a1d0-5055c8b475eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:34.314853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ee7e-9e49-4f49-a1d0-5055c8b475eb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:34.315132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ee7e-9e49-4f49-a1d0-5055c8b475eb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:34.315407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4ee7e-9e49-4f49-a1d0-5055c8b475eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:34.316664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ee7e-9e49-4f49-a1d0-5055c8b475eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:34.318176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4ee7e-9e49-4f49-a1d0-5055c8b475eb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:34.320087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 29/56] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:34.326253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:34.548139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09836f27-18d9-4f1f-89b1-4ae144ebae63 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:34.583073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c73372-460b-4318-aadb-a93299b9c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:34.583696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73372-460b-4318-aadb-a93299b9c026 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:34.583980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73372-460b-4318-aadb-a93299b9c026 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:34.584255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73372-460b-4318-aadb-a93299b9c026 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:34.584952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c73372-460b-4318-aadb-a93299b9c026 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:34.586523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c73372-460b-4318-aadb-a93299b9c026 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:34.617095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 28/57] 10.222.0.43 () {60 vars in 1140 bytes} [Tue Sep 13 17:46:34 2022] POST /identity/v3/projects => generated 298 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:36.814963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a150072-1b3c-49b3-bd7d-9ee566440487 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:36.815509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a150072-1b3c-49b3-bd7d-9ee566440487 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:36.815509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a150072-1b3c-49b3-bd7d-9ee566440487 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:36.816918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 30/58] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:36.822317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048457-01df-4ac7-9acd-2189916f09ad None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:36.822740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048457-01df-4ac7-9acd-2189916f09ad None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:36.822903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9048457-01df-4ac7-9acd-2189916f09ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:36.842219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a9048457-01df-4ac7-9acd-2189916f09ad None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:36.843437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a9048457-01df-4ac7-9acd-2189916f09ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:36.889286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:36.915894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 29/59] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:37.183901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a150072-1b3c-49b3-bd7d-9ee566440487 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:37.265110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:37.291256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a150072-1b3c-49b3-bd7d-9ee566440487 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:37.330023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295ffa9-f506-41e4-b9b0-5909aad75cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:37.330652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295ffa9-f506-41e4-b9b0-5909aad75cfe None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:37.330926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295ffa9-f506-41e4-b9b0-5909aad75cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:37.331190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295ffa9-f506-41e4-b9b0-5909aad75cfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:37.332371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295ffa9-f506-41e4-b9b0-5909aad75cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:37.333860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8295ffa9-f506-41e4-b9b0-5909aad75cfe None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:37.335455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 31/60] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:37.341323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048457-01df-4ac7-9acd-2189916f09ad None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:37.441120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9048457-01df-4ac7-9acd-2189916f09ad None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:37.475413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd8d80-d8b0-4a8c-b708-54642006dc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:37.476063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8d80-d8b0-4a8c-b708-54642006dc82 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:37.476345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8d80-d8b0-4a8c-b708-54642006dc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:37.476644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd8d80-d8b0-4a8c-b708-54642006dc82 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:37.477299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8d80-d8b0-4a8c-b708-54642006dc82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:37.478749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd8d80-d8b0-4a8c-b708-54642006dc82 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:37.507570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 30/61] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:46:37 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:40.853196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc373932-951c-4aa7-a85e-de3fe95c0218 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:40.853766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc373932-951c-4aa7-a85e-de3fe95c0218 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:40.853766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc373932-951c-4aa7-a85e-de3fe95c0218 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:40.855408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 32/62] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:40.861024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdecf52-d316-465f-a75c-b5955daf2c23 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:40.861873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdecf52-d316-465f-a75c-b5955daf2c23 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:40.862330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdecf52-d316-465f-a75c-b5955daf2c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:40.887110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9cdecf52-d316-465f-a75c-b5955daf2c23 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:40.888254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f42aa-6ba4-43a4-8012-71a8b4c11365 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:40.888740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9cdecf52-d316-465f-a75c-b5955daf2c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:40.888874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f42aa-6ba4-43a4-8012-71a8b4c11365 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:40.889318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f42aa-6ba4-43a4-8012-71a8b4c11365 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:40.891149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 33/63] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:40.906287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e0daf-67d7-4fb7-b792-72133fbeb423 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:40.906859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e0daf-67d7-4fb7-b792-72133fbeb423 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:40.907302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e0daf-67d7-4fb7-b792-72133fbeb423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:40.924519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:40.953621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-062e0daf-67d7-4fb7-b792-72133fbeb423 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:40.955264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-062e0daf-67d7-4fb7-b792-72133fbeb423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:41.006575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 31/64] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:41.082811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 34/65] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 17:46:41.280013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdecf52-d316-465f-a75c-b5955daf2c23 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:41.307622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19eef9-fbd5-4326-930b-b45e095178af None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.308265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19eef9-fbd5-4326-930b-b45e095178af None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.308757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19eef9-fbd5-4326-930b-b45e095178af None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.310471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 35/66] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:41.317002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2c48-7bcf-4459-9c28-44462bb26999 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.317653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2c48-7bcf-4459-9c28-44462bb26999 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.318246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2c48-7bcf-4459-9c28-44462bb26999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.352216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-edaf2c48-7bcf-4459-9c28-44462bb26999 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:41.354313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-edaf2c48-7bcf-4459-9c28-44462bb26999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:41.405381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdecf52-d316-465f-a75c-b5955daf2c23 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:41.442611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad8906-06ee-45ed-8d24-24b281c92bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:41.443266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad8906-06ee-45ed-8d24-24b281c92bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.443552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad8906-06ee-45ed-8d24-24b281c92bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.443815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad8906-06ee-45ed-8d24-24b281c92bb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.456742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad8906-06ee-45ed-8d24-24b281c92bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:41.459931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 36/67] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:41.461246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2c48-7bcf-4459-9c28-44462bb26999 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fcad8906-06ee-45ed-8d24-24b281c92bb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:41.462422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:41.463302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:41.464054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 32/68] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:41.467400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb36e-fa05-4fc3-bfbd-352184903ca0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.467766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb36e-fa05-4fc3-bfbd-352184903ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.468121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bb36e-fa05-4fc3-bfbd-352184903ca0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.474673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 33/69] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:41.475710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bb36e-fa05-4fc3-bfbd-352184903ca0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:41.479572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:41.512319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:41.588049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2c48-7bcf-4459-9c28-44462bb26999 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:41.609870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bb36e-fa05-4fc3-bfbd-352184903ca0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:41.627050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c550-0c5d-47ec-985f-454b2b9774e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:41.627958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c550-0c5d-47ec-985f-454b2b9774e3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.628318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c550-0c5d-47ec-985f-454b2b9774e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.628556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c550-0c5d-47ec-985f-454b2b9774e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.639990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c550-0c5d-47ec-985f-454b2b9774e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f6e1c550-0c5d-47ec-985f-454b2b9774e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:41.645010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:41.645865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 37/70] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:41.649169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852beba0-4211-4673-9825-7b24c492261a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.649861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852beba0-4211-4673-9825-7b24c492261a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.649861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852beba0-4211-4673-9825-7b24c492261a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.651905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b796-0303-4764-a17b-3384ceb22a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:41.652734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b796-0303-4764-a17b-3384ceb22a2a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.653028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b796-0303-4764-a17b-3384ceb22a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.653310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b796-0303-4764-a17b-3384ceb22a2a None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.654035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b796-0303-4764-a17b-3384ceb22a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:41.654342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b796-0303-4764-a17b-3384ceb22a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:41.657052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b796-0303-4764-a17b-3384ceb22a2a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:41.666570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-852beba0-4211-4673-9825-7b24c492261a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:41.667784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-852beba0-4211-4673-9825-7b24c492261a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:41.672334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 34/71] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:41.673716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c815d37-4ccb-4a08-aba1-63a3cc5ca583 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.674050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c815d37-4ccb-4a08-aba1-63a3cc5ca583 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.674338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c815d37-4ccb-4a08-aba1-63a3cc5ca583 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.675743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 35/72] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:41.676369 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c815d37-4ccb-4a08-aba1-63a3cc5ca583 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:41.741825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:41.751032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 38/73] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:41.751933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852beba0-4211-4673-9825-7b24c492261a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:41.778030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:41.794695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c815d37-4ccb-4a08-aba1-63a3cc5ca583 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:41.829083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c530c-7859-4b2b-8eec-f0be8f46c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:41.829697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c530c-7859-4b2b-8eec-f0be8f46c22f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.829975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c530c-7859-4b2b-8eec-f0be8f46c22f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.830246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c530c-7859-4b2b-8eec-f0be8f46c22f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.830927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c530c-7859-4b2b-8eec-f0be8f46c22f None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:41.831227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c530c-7859-4b2b-8eec-f0be8f46c22f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:41.832658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c530c-7859-4b2b-8eec-f0be8f46c22f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:41.841898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 36/74] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:41.842879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13979f2a-885b-496b-a242-85b779402277 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.843165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13979f2a-885b-496b-a242-85b779402277 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.843429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13979f2a-885b-496b-a242-85b779402277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.862119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-13979f2a-885b-496b-a242-85b779402277 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:41.863390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13979f2a-885b-496b-a242-85b779402277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:41.870077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852beba0-4211-4673-9825-7b24c492261a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:41.907294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:41.907954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:41.908223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:41.908531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] PATH_INFO: `/v3/users/bb561c12b9c44bd29f9bae620dc33608` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:41.909807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] RBAC: Authorizing `identity:get_user(user_id=bb561c12b9c44bd29f9bae620dc33608)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:41.911325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:41.913065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 39/75] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:41 2022] GET /identity/v3/users/bb561c12b9c44bd29f9bae620dc33608 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:41.913996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:41.952145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 37/76] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:41.954141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13979f2a-885b-496b-a242-85b779402277 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:41.984287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.031211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e977c-5044-4bb5-bd3c-653872785f55 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.074027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.074674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.074951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.075219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.082176 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.088757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:42.095515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:42.096734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:42.097504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 40/77] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:46:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:42.103514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.109103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13979f2a-885b-496b-a242-85b779402277 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.145524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.146184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.146470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.146738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.164317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.165865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.167643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 38/78] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:41 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:42.168502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.212509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94f988-58e3-42d7-935d-f6fabc0a7f64 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.236315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.252927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.253559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.253841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.254114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.274632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83d014-f4fc-4fcc-94ee-946afa4d8c6e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.308352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.309026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.309301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.309573 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.312313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.321484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:42.324313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.325875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:42.326395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:42.327326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:42.328075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 39/79] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:42.328741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.440690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342d7ab-35b7-4bf6-9e28-f1339f948012 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.448566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 41/80] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:46:42.449438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.476088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.476398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.476713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.476994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.477265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:42.478247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.479748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.488108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 40/81] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:42.488969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.547432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.558218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42266857-6689-483d-979d-d7acfcd76393 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.592748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdc1be-8c00-472d-bba7-523ecf421cad None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.595425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.596049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.596325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.596623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.629189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.629839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.630122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.630399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.631561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.633151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.636525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 41/82] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:42.636525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.667184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.668724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.694556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.743000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756f7f5-8372-4df0-b52b-901638ef4848 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.772748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.775441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 42/83] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:46:42.776231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.785748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.786371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.786652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.786921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.787594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:42.787896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.789385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.799015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 42/84] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:42.800143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.885097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e265cb-562b-4664-ad75-c998711951bb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.909240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:42.916537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b5129-7277-4b74-b239-221421d9b8e3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:42.922783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.923423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.923709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.924378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.925550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.927706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.929492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 43/85] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:42.930253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:42.952655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:42.953192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:42.953464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:42.953732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:42.954912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:42.956398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:42.958044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 43/86] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:42.958819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:43.020244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:43.049502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396adef-881a-4860-950f-0eb3067765f7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:43.069361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322097b-a08e-4854-a110-36e3d144ba62 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:43.084180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:43.084843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:43.085251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:43.085561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:43.086213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:43.087646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:43.105716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:43.106471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:43.106753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:43.106894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:43.108085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:43.109711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:43.111366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 44/87] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:42 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:43.112269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:43.115375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 44/88] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:46:42 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:43.116200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:43.185317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:43.251010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a34ca-588c-4cce-af83-b43592779b8d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:43.254229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361f756-d55a-406f-9f9d-2b05303b2d72 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:43.289231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:43.293606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:43.294249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:43.294533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:43.294811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:43.295596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=bb561c12b9c44bd29f9bae620dc33608) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:43.295889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:43.297614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:43.304477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 45/89] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:46:43 2022] GET /identity/v3/role_assignments?user.id=bb561c12b9c44bd29f9bae620dc33608&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:43.305571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:43.306351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:43.306777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:43.307167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:43.309121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:43.311491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:43.315775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:43.316047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 45/90] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:43 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:43.316047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:43.481781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04c8e8-24de-4bb1-8cdd-169ae4eb35bd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:43.512515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0bf2e-0a26-4cb1-9318-62a741a2fc4f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:43.527124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:43.564219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:43.564917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:43.565066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:43.565241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:43.565509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:43.566027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:43.566493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:43.566958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:43.568395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=2fd61952074f49b9bed58536e54de503) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:43.568953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:43.571750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:43.582996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 46/91] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:46:43 2022] GET /identity/v3/role_assignments?user.id=2fd61952074f49b9bed58536e54de503&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=5c592b5b170d48b0825640b8283425de => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:43.583993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:43.681123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:43.736116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:43.736887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:43.789696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66efa-17d1-46d6-9a91-07c704d13d97 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:43.870617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:43.871561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:43.873162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:43.873460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:43.975183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 46/92] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:46:43.986521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:44.005247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:44.028458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:44.028458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:44.098191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:44.273157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 47/93] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:46:44.279678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:44.305705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a1e79-f391-4641-8395-e09f125b499e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:44.437284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:44.438197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:44.438670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:44.439125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:44.445609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:44.448280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:44.451333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 47/94] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:43 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:44.461805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:44.604991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715d1a3-44f2-4527-a885-1e8dba020aab None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:44.657380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:44.701303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:44.701892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:44.702173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:44.702447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] PATH_INFO: `/v3/projects/16188004a08942a0a5352ad3d555aff7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:44.703671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] RBAC: Authorizing `identity:get_project(project_id=16188004a08942a0a5352ad3d555aff7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:44.705175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3613d-c118-4876-8382-5d304bbc8d0b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:44.705291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:44.707231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 48/95] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:44 2022] GET /identity/v3/projects/16188004a08942a0a5352ad3d555aff7 => generated 312 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:44.714362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:44.785636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:44.791884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201e11c-32d1-4371-89f8-f4ef1915a25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:44.793619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201e11c-32d1-4371-89f8-f4ef1915a25b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:44.794358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201e11c-32d1-4371-89f8-f4ef1915a25b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:44.795064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201e11c-32d1-4371-89f8-f4ef1915a25b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:44.797101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201e11c-32d1-4371-89f8-f4ef1915a25b None admin] RBAC: Adding query filter params (role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=2fd61952074f49b9bed58536e54de503) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:44.798502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201e11c-32d1-4371-89f8-f4ef1915a25b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:44.802026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201e11c-32d1-4371-89f8-f4ef1915a25b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:44.815023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 48/96] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:46:44 2022] GET /identity/v3/role_assignments?user.id=2fd61952074f49b9bed58536e54de503&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 259 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:44.931522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f64a43-3be6-4907-a6f7-5afd5c8d098d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:45.001745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2897937-c73a-42a5-925a-432afa4c0a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:45.003125 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897937-c73a-42a5-925a-432afa4c0a6d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:45.003790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897937-c73a-42a5-925a-432afa4c0a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:45.004536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2897937-c73a-42a5-925a-432afa4c0a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:45.006103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2897937-c73a-42a5-925a-432afa4c0a6d None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=16188004a08942a0a5352ad3d555aff7, user.id=2fd61952074f49b9bed58536e54de503) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:45.006849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2897937-c73a-42a5-925a-432afa4c0a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:45.009928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2897937-c73a-42a5-925a-432afa4c0a6d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:45.021484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 49/97] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:46:44 2022] GET /identity/v3/role_assignments?user.id=2fd61952074f49b9bed58536e54de503&scope.project.id=16188004a08942a0a5352ad3d555aff7&role.id=5c592b5b170d48b0825640b8283425de => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:46.489054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b7128-87b2-4ccf-ad24-0cee0809b86b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:46.490574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b7128-87b2-4ccf-ad24-0cee0809b86b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:46.491382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b7128-87b2-4ccf-ad24-0cee0809b86b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:46.494510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 49/98] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:46.500890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f307a6-d922-4bba-9c99-ed4289e380ce None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:46.502301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f307a6-d922-4bba-9c99-ed4289e380ce None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:46.503048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f307a6-d922-4bba-9c99-ed4289e380ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:46.541415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b6f307a6-d922-4bba-9c99-ed4289e380ce None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:46.543856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b6f307a6-d922-4bba-9c99-ed4289e380ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:46.698888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 50/99] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:46.836968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d7808-97ff-4f4d-9d26-c137fcabb70d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:46.837947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d7808-97ff-4f4d-9d26-c137fcabb70d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:46.838704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d7808-97ff-4f4d-9d26-c137fcabb70d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:46.841606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 50/100] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:46.848795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825793-d608-46e0-ae71-720b64892ccf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:46.849736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825793-d608-46e0-ae71-720b64892ccf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:46.850465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825793-d608-46e0-ae71-720b64892ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:46.893383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3f825793-d608-46e0-ae71-720b64892ccf None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:46.896066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3f825793-d608-46e0-ae71-720b64892ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:46.933182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:47.071731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 51/101] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:47.211297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db12f-c106-419e-85c5-4b863a7921f8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:47.212299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db12f-c106-419e-85c5-4b863a7921f8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:47.213091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db12f-c106-419e-85c5-4b863a7921f8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:47.216065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 51/102] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:47.223264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80854cb4-9c20-48d9-9e5b-5ab4a68f2af7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:47.224312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80854cb4-9c20-48d9-9e5b-5ab4a68f2af7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:47.226009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25db12f-c106-419e-85c5-4b863a7921f8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:47.227882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80854cb4-9c20-48d9-9e5b-5ab4a68f2af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:47.282196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-80854cb4-9c20-48d9-9e5b-5ab4a68f2af7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:47.286325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-80854cb4-9c20-48d9-9e5b-5ab4a68f2af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:47.468353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 52/103] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:47.469252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80854cb4-9c20-48d9-9e5b-5ab4a68f2af7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:47.484986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25db12f-c106-419e-85c5-4b863a7921f8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:47.581448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1f0cf-7da5-4811-9fc4-0b02b8d879f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:47.582379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f0cf-7da5-4811-9fc4-0b02b8d879f3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:47.582767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f0cf-7da5-4811-9fc4-0b02b8d879f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:47.583137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1f0cf-7da5-4811-9fc4-0b02b8d879f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:47.617573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1f0cf-7da5-4811-9fc4-0b02b8d879f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-92d1f0cf-7da5-4811-9fc4-0b02b8d879f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:47.623247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:47.624184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:46:47.624978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:47.627583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 52/104] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:47.629139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba827e-6b6f-4c43-b087-8f25b186abf5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:47.629414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba827e-6b6f-4c43-b087-8f25b186abf5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:47.629705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba827e-6b6f-4c43-b087-8f25b186abf5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:47.631231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 53/105] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:47.638484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba827e-6b6f-4c43-b087-8f25b186abf5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:47.646923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:47.673200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:47.737901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80854cb4-9c20-48d9-9e5b-5ab4a68f2af7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:47.811657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4a4d3-4c49-46e5-91e8-581270849ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:47.812338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a4d3-4c49-46e5-91e8-581270849ede None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:47.812646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a4d3-4c49-46e5-91e8-581270849ede None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:47.812923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4a4d3-4c49-46e5-91e8-581270849ede None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:47.849404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4a4d3-4c49-46e5-91e8-581270849ede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-44c4a4d3-4c49-46e5-91e8-581270849ede None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:47.857014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:47.858804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:47.860713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:47.862109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 53/106] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:47.868141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ae9d7-1a18-4df7-a42a-6b10e862e75e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:47.868141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ae9d7-1a18-4df7-a42a-6b10e862e75e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:47.868141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ae9d7-1a18-4df7-a42a-6b10e862e75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:47.889940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a15ae9d7-1a18-4df7-a42a-6b10e862e75e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:47.891530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a15ae9d7-1a18-4df7-a42a-6b10e862e75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:47.909265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba827e-6b6f-4c43-b087-8f25b186abf5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:47.987366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:47.987922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:47.988208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:47.988510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:47.989217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:47.989524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:47.991012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:48.015284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 54/107] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:48.021502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:48.050602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 54/108] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:48.056079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ae9d7-1a18-4df7-a42a-6b10e862e75e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:48.102828 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:48.172822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5298519-3dd6-417c-9e81-5fba131e64b9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:48.217815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:48.245411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:48.245971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:48.246258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:48.246532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:48.247247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:48.247557 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:48.249104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:48.264570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 55/109] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:48.265625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:48.301780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ae9d7-1a18-4df7-a42a-6b10e862e75e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:48.409901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:48.410598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:48.410883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:48.411158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] PATH_INFO: `/v3/users/f13c21a275964533844c82cd020a6752` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:48.423389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d38650-8e95-471e-8b8c-d1342acd2e97 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:48.444846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] RBAC: Authorizing `identity:get_user(user_id=f13c21a275964533844c82cd020a6752)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:48.446693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:48.457046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 55/110] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:48 2022] GET /identity/v3/users/f13c21a275964533844c82cd020a6752 => generated 289 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:48.458463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:48.481973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:48.493107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:48.493781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:48.494073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:48.494350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:48.496034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:48.497556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:48.500023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 56/111] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:48.501347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:48.610871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb4f18-5fa2-4848-9b39-81a5a269dc5c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:48.642175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a70c677-f6c3-4ffe-af3c-d517810c5b3c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:48.651519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:48.668221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:48.669274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:48.669769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:48.670222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:48.672882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:48.676286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:48.680654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 56/112] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:48 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:48.682739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:48.691933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:48.692760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:48.693115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:48.693467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:48.782809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:48.794385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:48.796588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:48.826179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330809b8-e6e8-4079-94cd-3acb6f3fa224 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:48.877051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:48.878037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:48.878500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:48.878946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:48.897810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:48.906284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:48.907216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:48.907973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:48.914038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 57/113] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:46:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:48.914038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:48.955027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 57/114] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:46:48.956807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:49.005099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:49.069194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa00dc-256d-468e-8e12-a05aa3479394 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:49.096731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:49.105020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752242ac-e03a-4691-aba3-44761b8dcbb8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:49.120544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150795-532f-4240-920b-905608dae92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:49.121923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150795-532f-4240-920b-905608dae92a None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.122346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150795-532f-4240-920b-905608dae92a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.123007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97150795-532f-4240-920b-905608dae92a None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.123828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97150795-532f-4240-920b-905608dae92a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:49.126061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97150795-532f-4240-920b-905608dae92a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:49.148488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ccebc-28e5-4f62-947a-e77db8d5109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:49.149516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ccebc-28e5-4f62-947a-e77db8d5109f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.149988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ccebc-28e5-4f62-947a-e77db8d5109f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.150342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ccebc-28e5-4f62-947a-e77db8d5109f None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.152113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ccebc-28e5-4f62-947a-e77db8d5109f None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:49.154750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ccebc-28e5-4f62-947a-e77db8d5109f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:49.157607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 58/115] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:48 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:49.159611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39d654-74e1-454f-a82f-b7b24f20e694 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.160046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39d654-74e1-454f-a82f-b7b24f20e694 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.165890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39d654-74e1-454f-a82f-b7b24f20e694 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.166901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 59/116] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:49.168308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39d654-74e1-454f-a82f-b7b24f20e694 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:49.171330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 58/117] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:46:48 2022] POST /identity/v3/projects => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:49.173762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150795-532f-4240-920b-905608dae92a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:49.365810 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39d654-74e1-454f-a82f-b7b24f20e694 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:49.370511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97150795-532f-4240-920b-905608dae92a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:49.382655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:49.420691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:49.421569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.422011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.422447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.423499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:49.423990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:49.426224 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:49.432416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b2328-a666-4591-8be0-4b44b58d3f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:49.433334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2328-a666-4591-8be0-4b44b58d3f59 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.433818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2328-a666-4591-8be0-4b44b58d3f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.434230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b2328-a666-4591-8be0-4b44b58d3f59 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.435697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b2328-a666-4591-8be0-4b44b58d3f59 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:49.437474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b2328-a666-4591-8be0-4b44b58d3f59 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:49.445957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 60/118] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:49.445957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:49.446328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 59/119] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:49 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:49.446328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1af1-661d-408c-9a4f-e569999caeb2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.446544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1af1-661d-408c-9a4f-e569999caeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.446899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1af1-661d-408c-9a4f-e569999caeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.469628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b32b1af1-661d-408c-9a4f-e569999caeb2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:49.470869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b32b1af1-661d-408c-9a4f-e569999caeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:49.482098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:49.578094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 60/120] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:49.583183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4cd70-4432-4636-892f-99db15ded8f3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.583665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4cd70-4432-4636-892f-99db15ded8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.584138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4cd70-4432-4636-892f-99db15ded8f3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.586744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 61/121] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:49.589006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4cd70-4432-4636-892f-99db15ded8f3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:49.609541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe75fe7-ebf4-432c-8d36-fb488a17ebc4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:49.685333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:49.688680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.689158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.689702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383/users/2fd61952074f49b9bed58536e54de503/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.694882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bade20834645af2806f86402c383, user_id=2fd61952074f49b9bed58536e54de503, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:49.698118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:49.753005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 61/122] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:46:49 2022] PUT /identity/v3/projects/a5a5bade20834645af2806f86402c383/users/2fd61952074f49b9bed58536e54de503/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:46:49.756579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:49.832823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4cd70-4432-4636-892f-99db15ded8f3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:49.833629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:49.899823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d675f8-3b28-4e51-847c-e781e4a89218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:49.901098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d675f8-3b28-4e51-847c-e781e4a89218 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.901582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d675f8-3b28-4e51-847c-e781e4a89218 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.902049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d675f8-3b28-4e51-847c-e781e4a89218 None admin] PATH_INFO: `/v3/users/bb561c12b9c44bd29f9bae620dc33608` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.904809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d675f8-3b28-4e51-847c-e781e4a89218 None admin] RBAC: Authorizing `identity:get_user(user_id=bb561c12b9c44bd29f9bae620dc33608)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:49.907904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d675f8-3b28-4e51-847c-e781e4a89218 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:49.911331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 62/123] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:49 2022] GET /identity/v3/users/bb561c12b9c44bd29f9bae620dc33608 => generated 247 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:49.917797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da9950-81bb-4a2c-96c1-66c1a487502d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:49.918260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da9950-81bb-4a2c-96c1-66c1a487502d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:49.919254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da9950-81bb-4a2c-96c1-66c1a487502d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:49.952206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f6da9950-81bb-4a2c-96c1-66c1a487502d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:49.955337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f6da9950-81bb-4a2c-96c1-66c1a487502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:50.026067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:50.078369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b207-30fb-41db-a64a-3791a3d9395a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:50.091409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 63/124] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:50.093012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da9950-81bb-4a2c-96c1-66c1a487502d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:50.180368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:50.181511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:50.181980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:50.182461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:50.183885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] RBAC: Adding query filter params (role.id=89cf360b7f5f43ec85bcf8dea35985dd, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=f13c21a275964533844c82cd020a6752) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:50.184471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:50.187106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:50.198911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 62/125] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:46:49 2022] GET /identity/v3/role_assignments?user.id=f13c21a275964533844c82cd020a6752&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=89cf360b7f5f43ec85bcf8dea35985dd => generated 259 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:50.200554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:50.401610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da9950-81bb-4a2c-96c1-66c1a487502d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:50.405306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:50.494705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ec66-4ad9-4a06-bde8-7bd441d6717d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:50.522763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:50.525268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:50.526191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:50.526935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:50.529976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:50.540465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:50.545290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 64/126] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:50 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:50.554232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:50.630535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:50.631608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:50.632144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:50.632634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:50.670120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:50.688802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:50.691476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 63/127] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 491 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:50.694559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:50.749286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:50.790212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486937e0-8ce1-422a-bbff-95845b867bd1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:50.850611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:50.851568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:50.851568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:50.851844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383/users/bb561c12b9c44bd29f9bae620dc33608/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:50.856397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bade20834645af2806f86402c383, user_id=bb561c12b9c44bd29f9bae620dc33608, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:50.859147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:50.887520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 65/128] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:46:50 2022] PUT /identity/v3/projects/a5a5bade20834645af2806f86402c383/users/bb561c12b9c44bd29f9bae620dc33608/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 17:46:50.888832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:50.958467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:51.008843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ce1d2-dd0f-4474-82d2-d84c679b8797 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:51.082663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:51.083341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:51.083617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:51.084057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:51.109356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:51.114603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71e485-d7ca-4073-9898-c4aa061f4fee None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:51.119413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:51.120934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:51.122434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 64/129] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:46:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:51.132416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:51.212376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:51.215628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:51.215958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:51.216188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:51.218835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:51.218835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:51.218835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:51.228747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 66/130] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:51.234900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:51.258258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:51.344779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:51.355802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31100e8-2b97-41af-9bc4-a249b411a689 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:51.380177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedede32-4d04-477a-b1a6-b27afca74b0e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:51.395643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:51.396592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:51.397013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:51.397491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:51.399762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:51.401479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:51.403302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 67/131] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:51 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:51.409639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:51.531522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e733091-2bbf-4864-9e42-1ae76dc6d5e2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:51.534079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:51.535561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:51.535922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:51.536549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:51.537975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:51.538796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:51.549884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:51.572751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 65/132] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:46:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:51.575569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:51.576408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:51.576903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:51.577311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] PATH_INFO: `/v3/projects/16188004a08942a0a5352ad3d555aff7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:51.578725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] RBAC: Authorizing `identity:get_project(project_id=16188004a08942a0a5352ad3d555aff7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:51.580485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:51.581730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:51.583014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 68/133] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:51 2022] GET /identity/v3/projects/16188004a08942a0a5352ad3d555aff7 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:51.589534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:51.652976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:51.803878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03c745-d453-4f91-980d-fb7a7aacfe0b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:51.829263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b326e6-b3a8-4dbc-a37f-b8232aaac383 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:51.880382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:51.881234 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:51.881364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:51.881812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] PATH_INFO: `/v3/projects/16188004a08942a0a5352ad3d555aff7/users/2fd61952074f49b9bed58536e54de503/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:51.884867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] RBAC: Authorizing `identity:create_grant(project_id=16188004a08942a0a5352ad3d555aff7, user_id=2fd61952074f49b9bed58536e54de503, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:51.886380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:51.893867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:51.907363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:51.908563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:51.909102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:51.909568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:51.911955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:51.914969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:51.916900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 69/134] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:46:51 2022] PUT /identity/v3/projects/16188004a08942a0a5352ad3d555aff7/users/2fd61952074f49b9bed58536e54de503/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:46:51.918191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 66/135] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:51 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:51.927533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:52.113317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:52.155831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abefb1eb-e0e7-4de7-bca5-80f337866dcc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:52.227471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edec40-9c80-4c87-8993-bf14d1ce5350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:52.228720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edec40-9c80-4c87-8993-bf14d1ce5350 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:52.229239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edec40-9c80-4c87-8993-bf14d1ce5350 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:52.229731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edec40-9c80-4c87-8993-bf14d1ce5350 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:52.237499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4edec40-9c80-4c87-8993-bf14d1ce5350 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:52.239579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4edec40-9c80-4c87-8993-bf14d1ce5350 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:52.241794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 70/136] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:51 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:52.254519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:52.502082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efac43-b351-4f31-90b1-69882a86a37e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:52.585924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670b98e-163f-4bfb-9989-5b689f912289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:52.587557 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670b98e-163f-4bfb-9989-5b689f912289 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:52.588334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670b98e-163f-4bfb-9989-5b689f912289 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:52.589123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670b98e-163f-4bfb-9989-5b689f912289 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383/users/2fd61952074f49b9bed58536e54de503/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:52.597669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670b98e-163f-4bfb-9989-5b689f912289 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bade20834645af2806f86402c383, user_id=2fd61952074f49b9bed58536e54de503, role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:52.600964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670b98e-163f-4bfb-9989-5b689f912289 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:52.639130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 67/137] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:46:52 2022] PUT /identity/v3/projects/a5a5bade20834645af2806f86402c383/users/2fd61952074f49b9bed58536e54de503/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:46:53.532112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098125c-4786-477a-9e53-f6ee5cdcd6a0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:53.533770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098125c-4786-477a-9e53-f6ee5cdcd6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:53.534549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098125c-4786-477a-9e53-f6ee5cdcd6a0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:53.537592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 71/138] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:46:53.544938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa4bf-0277-4496-869d-e2677ee8f7b1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:53.546523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa4bf-0277-4496-869d-e2677ee8f7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:53.547308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa4bf-0277-4496-869d-e2677ee8f7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:53.588037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9adfa4bf-0277-4496-869d-e2677ee8f7b1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:53.590682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9adfa4bf-0277-4496-869d-e2677ee8f7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:53.718231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:53.758790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 68/139] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:54.308538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098125c-4786-477a-9e53-f6ee5cdcd6a0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:54.448670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098125c-4786-477a-9e53-f6ee5cdcd6a0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:54.494218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5def15b-00c5-4e27-b21a-c20edb36ec2d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:54.494564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5def15b-00c5-4e27-b21a-c20edb36ec2d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:54.494928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5def15b-00c5-4e27-b21a-c20edb36ec2d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:54.496492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 69/140] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:46:54.502870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34b11d-08ca-47a0-94e6-9c582df59c9a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:54.503207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34b11d-08ca-47a0-94e6-9c582df59c9a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:54.503423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34b11d-08ca-47a0-94e6-9c582df59c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:54.510126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:54.511245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:54.511738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:54.512231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:54.514339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:54.516360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:54.518463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 72/141] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:46:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:54.524718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:54.541437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4b34b11d-08ca-47a0-94e6-9c582df59c9a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:54.541437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b34b11d-08ca-47a0-94e6-9c582df59c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:54.581032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:54.722896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 70/142] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:54.745337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ceff7-4d55-441c-93bf-084eff978428 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:54.815950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b289936-2cdb-4067-a595-19a150527475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:54.817028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b289936-2cdb-4067-a595-19a150527475 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:54.817514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b289936-2cdb-4067-a595-19a150527475 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:54.818000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b289936-2cdb-4067-a595-19a150527475 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:54.818909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b289936-2cdb-4067-a595-19a150527475 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:54.820764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b289936-2cdb-4067-a595-19a150527475 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:54.867552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 73/143] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:46:54 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:46:54.953633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4b2e-95bb-4cd6-a9d2-e87bfaaa77e1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:54.954289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4b2e-95bb-4cd6-a9d2-e87bfaaa77e1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:54.954811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab4b2e-95bb-4cd6-a9d2-e87bfaaa77e1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:54.956923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 71/144] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:54.965315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7c7af-6d7b-4078-bbfe-76ea93257893 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:54.965952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7c7af-6d7b-4078-bbfe-76ea93257893 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:54.966429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7c7af-6d7b-4078-bbfe-76ea93257893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:55.002786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a0b7c7af-6d7b-4078-bbfe-76ea93257893 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:55.004393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a0b7c7af-6d7b-4078-bbfe-76ea93257893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:55.006383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276064a-6163-4a12-83cb-9083472afe50 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:55.007030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276064a-6163-4a12-83cb-9083472afe50 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:55.007512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276064a-6163-4a12-83cb-9083472afe50 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:55.009111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 72/145] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:46:55.015512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78f56-ab73-4486-a22a-2a3633a79437 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:55.016124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78f56-ab73-4486-a22a-2a3633a79437 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:55.016639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e78f56-ab73-4486-a22a-2a3633a79437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:55.046227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-90e78f56-ab73-4486-a22a-2a3633a79437 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:55.048071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90e78f56-ab73-4486-a22a-2a3633a79437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:55.109133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 74/146] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:55.156619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 73/147] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:46:55.259924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7c7af-6d7b-4078-bbfe-76ea93257893 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:55.274821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:55.379746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7c7af-6d7b-4078-bbfe-76ea93257893 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:55.420762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:55.421326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:55.421600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:55.421867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:55.435805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:55.441210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:55.442174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:55.442972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 75/148] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:46:55.449613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78f56-ab73-4486-a22a-2a3633a79437 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:55.503839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:55.532819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:55.651245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e78f56-ab73-4486-a22a-2a3633a79437 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:55.655334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:55.735494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d03ef0-7b6a-4a30-9ee6-6400849c8415 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:55.747693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:55.748615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:55.748615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:55.749073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:55.749642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:55.749964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:55.751473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:55.767513 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 74/149] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:46:55.768813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:55.809000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:55.810853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:55.811340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:55.811806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:55.837420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:55.845600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:55.847220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 76/150] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:46:55.848724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:55.928541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790dec8-fcab-4e34-b4e3-475525d13394 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:55.950509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:56.010373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:56.011919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.013080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.013927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.034927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:56.037119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332deed-f58a-4440-8101-1c393ff565cc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:56.046721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:56.048408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:46:56.050372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:56.055480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 75/151] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:46:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:56.061611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:56.103727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:56.134863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:56.136550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.137432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.138590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.145038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:56.148931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:56.156336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 77/152] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:55 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:56.159944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:56.331566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7ca2f-deb2-4f08-97b4-7aeaf8aed759 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:56.368971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc38422-21f6-4327-b625-cc05fd37a8d6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:56.381158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:56.382144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.382652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.383106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.384061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:56.384603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:56.387135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:56.400836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 76/153] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:56.406299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:56.454964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c99aa-17ec-4b41-acf0-f9cb987b6034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:56.456927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c99aa-17ec-4b41-acf0-f9cb987b6034 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.458313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c99aa-17ec-4b41-acf0-f9cb987b6034 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.458903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c99aa-17ec-4b41-acf0-f9cb987b6034 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.460089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c99aa-17ec-4b41-acf0-f9cb987b6034 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:56.462108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c99aa-17ec-4b41-acf0-f9cb987b6034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:56.463587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c99aa-17ec-4b41-acf0-f9cb987b6034 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:56.466387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:56.478977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 78/154] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:56.482020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e6756-696c-48e9-b340-768cf5a23eb4 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.482020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e6756-696c-48e9-b340-768cf5a23eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.482020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e6756-696c-48e9-b340-768cf5a23eb4 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.483068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 79/155] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:56.488041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e6756-696c-48e9-b340-768cf5a23eb4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:56.587687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:56.596920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b485042-7158-4cf9-83f5-a7f9a113c734 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:56.643830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148bcbc-8dd3-4dec-bc72-ff4d2c8a08ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:56.644575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bcbc-8dd3-4dec-bc72-ff4d2c8a08ba None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.644855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bcbc-8dd3-4dec-bc72-ff4d2c8a08ba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.645118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bcbc-8dd3-4dec-bc72-ff4d2c8a08ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.660446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e6756-696c-48e9-b340-768cf5a23eb4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:56.746709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148bcbc-8dd3-4dec-bc72-ff4d2c8a08ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:56.747251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1819e-2478-4d0d-a455-72eb9d007c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:56.748551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1819e-2478-4d0d-a455-72eb9d007c11 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.748843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148bcbc-8dd3-4dec-bc72-ff4d2c8a08ba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:56.749262 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1819e-2478-4d0d-a455-72eb9d007c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.749746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1819e-2478-4d0d-a455-72eb9d007c11 None admin] PATH_INFO: `/v3/users/f13c21a275964533844c82cd020a6752` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.752243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1819e-2478-4d0d-a455-72eb9d007c11 None admin] RBAC: Authorizing `identity:get_user(user_id=f13c21a275964533844c82cd020a6752)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:56.755682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1819e-2478-4d0d-a455-72eb9d007c11 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:56.764263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 80/156] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:56 2022] GET /identity/v3/users/f13c21a275964533844c82cd020a6752 => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:56.764263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431feeac-daf5-454b-a390-f78dbb104703 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:56.764263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431feeac-daf5-454b-a390-f78dbb104703 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:56.764263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431feeac-daf5-454b-a390-f78dbb104703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:56.812260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-431feeac-daf5-454b-a390-f78dbb104703 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:56.814801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-431feeac-daf5-454b-a390-f78dbb104703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:56.822701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:56.998741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 81/157] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:56.999938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431feeac-daf5-454b-a390-f78dbb104703 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:57.056032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 77/158] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:46:57.057484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54fdc4-1319-4796-b379-815ae95c683c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:57.057766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54fdc4-1319-4796-b379-815ae95c683c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:57.058069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54fdc4-1319-4796-b379-815ae95c683c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:57.059415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 78/159] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:46:57.060226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54fdc4-1319-4796-b379-815ae95c683c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:57.187643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:57.309183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54fdc4-1319-4796-b379-815ae95c683c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:57.320457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431feeac-daf5-454b-a390-f78dbb104703 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:57.369835 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:57.381165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:57.382323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:57.382430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071dc5a-5c3e-4d29-94fa-029bf324c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:57.382887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:57.382997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071dc5a-5c3e-4d29-94fa-029bf324c044 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:57.383236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071dc5a-5c3e-4d29-94fa-029bf324c044 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:57.383506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071dc5a-5c3e-4d29-94fa-029bf324c044 None admin] PATH_INFO: `/v3/users/bb561c12b9c44bd29f9bae620dc33608` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:57.383795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:57.388451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071dc5a-5c3e-4d29-94fa-029bf324c044 None admin] RBAC: Authorizing `identity:get_user(user_id=bb561c12b9c44bd29f9bae620dc33608)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:57.388451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071dc5a-5c3e-4d29-94fa-029bf324c044 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:57.388451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 82/160] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:56 2022] GET /identity/v3/users/bb561c12b9c44bd29f9bae620dc33608 => generated 247 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:57.389681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eade60-0d33-4614-a2ab-d2b94bfaabed None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:57.389958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eade60-0d33-4614-a2ab-d2b94bfaabed None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:57.390268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eade60-0d33-4614-a2ab-d2b94bfaabed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:57.390524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:57.392257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:57.394791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 79/161] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:57 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:57.396081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:57.416514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e7eade60-0d33-4614-a2ab-d2b94bfaabed None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:46:57.420083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e7eade60-0d33-4614-a2ab-d2b94bfaabed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:46:57.538554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 83/162] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:46:57.539232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eade60-0d33-4614-a2ab-d2b94bfaabed None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:57.631494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13cfb0-9b27-45c4-9f08-262f4190c219 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:57.675094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:57.689521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:57.692022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:57.693339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:57.694670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:57.702028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:57.704647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:57.710542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 80/163] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:57 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:57.712709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:57.799287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eade60-0d33-4614-a2ab-d2b94bfaabed None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:57.826819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:57.878092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:57.879092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:57.879533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:57.879997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:57.895477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffe7a-3a10-4b9e-98b9-0b1d9e474855 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:57.921555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:57.931196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:57.933214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:46:57.935007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:46:57.946723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 84/164] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:46:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:57.948487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:57.953291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:57.953948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:57.954230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:57.954500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:58.073082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:58.073082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:58.233271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:58.328482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 81/165] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:46:58.334134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:58.468105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced589f-8446-467e-bc86-5075ed78f330 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:58.547785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:58.564600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef7e65-a2ed-4bf5-8bee-a746bc4b4704 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:58.586117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:58.586822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:58.587104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:58.587343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383/users/f13c21a275964533844c82cd020a6752/roles/89cf360b7f5f43ec85bcf8dea35985dd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:58.589894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bade20834645af2806f86402c383, user_id=f13c21a275964533844c82cd020a6752, role_id=89cf360b7f5f43ec85bcf8dea35985dd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:58.591401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:58.619076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 85/166] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:46:57 2022] PUT /identity/v3/projects/a5a5bade20834645af2806f86402c383/users/f13c21a275964533844c82cd020a6752/roles/89cf360b7f5f43ec85bcf8dea35985dd => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:46:58.621171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:58.654680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:58.656998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:58.657300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:58.657573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:58.658367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=2fd61952074f49b9bed58536e54de503) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:58.658664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:58.663456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:58.677223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 82/167] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:46:58 2022] GET /identity/v3/role_assignments?user.id=2fd61952074f49b9bed58536e54de503&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=5c592b5b170d48b0825640b8283425de => generated 608 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:58.678118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:58.744708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:58.873165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2590a3-eeaa-4d1e-b1a0-1a526fd055b2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:58.967307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:58.968924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:58.969644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:58.970340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:58.977817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:58.981158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:58.985027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 83/168] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:46:58 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:58.986826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:58.992468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940279b-71f9-4265-adfa-e718d752fefe None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:59.092565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:59.093209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:59.093485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:59.093749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:59.094439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:59.094739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:59.096169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:59.122864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 86/169] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:46:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:59.123958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:59.233822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:59.240762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4da3-5f08-4b58-979e-e21dcba0d977 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:59.250757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:46:59.307740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:59.308545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:59.308816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:59.309078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:59.342110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:46:59.361523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:46:59.363555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:46:59.365458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:46:59.369289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 84/170] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:46:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:46:59.375873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:59.492179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb011b-0063-49bb-bb18-8063c637959f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:59.646121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:59.647057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:59.647604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:59.648094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:59.649524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=bb561c12b9c44bd29f9bae620dc33608) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:46:59.650288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:59.653307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:59.655739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca373d3a-dc2f-4a4f-80fe-480657232753 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:46:59.672141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 87/171] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:46:59 2022] GET /identity/v3/role_assignments?user.id=bb561c12b9c44bd29f9bae620dc33608&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 549 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:59.673970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:59.754561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:46:59.755672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:46:59.756167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:46:59.756772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:46:59.760491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:46:59.763870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:46:59.772018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 85/172] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:46:59 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:46:59.774109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:46:59.853932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:00.031821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a48b6-5949-4d4a-9fa3-87de3dd5ce33 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:00.057280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5de64-c729-453b-a8bf-6c8fdb165297 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:00.067146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:00.097053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aec2a3-1aa5-4984-8d77-5c9eb3f42f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:00.098524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec2a3-1aa5-4984-8d77-5c9eb3f42f23 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:00.099188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec2a3-1aa5-4984-8d77-5c9eb3f42f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:00.099805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec2a3-1aa5-4984-8d77-5c9eb3f42f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:00.197812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:00.198857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:00.199355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:00.199831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:00.201159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:00.201713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:00.204527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:00.231983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 88/173] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:46:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 559 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:00.238656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:00.253631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aec2a3-1aa5-4984-8d77-5c9eb3f42f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:00.257061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aec2a3-1aa5-4984-8d77-5c9eb3f42f23 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:00.333240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:00.520586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 86/174] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:46:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:00.523504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7760818-4aeb-48d4-83d3-3a7b0beac73d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:00.524020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7760818-4aeb-48d4-83d3-3a7b0beac73d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:00.524542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7760818-4aeb-48d4-83d3-3a7b0beac73d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:00.528286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 87/175] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:00.532298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7760818-4aeb-48d4-83d3-3a7b0beac73d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:00.658627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650088c-ce5a-4244-84bb-6d458bcb523e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:00.774015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e520e-5bb6-4915-9d55-c8d08216a211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:00.775384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e520e-5bb6-4915-9d55-c8d08216a211 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:00.775855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e520e-5bb6-4915-9d55-c8d08216a211 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:00.776257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e520e-5bb6-4915-9d55-c8d08216a211 None admin] PATH_INFO: `/v3/users/2fd61952074f49b9bed58536e54de503` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:00.781580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e520e-5bb6-4915-9d55-c8d08216a211 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd61952074f49b9bed58536e54de503)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:00.784582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e520e-5bb6-4915-9d55-c8d08216a211 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:00.787767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 89/176] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:00 2022] GET /identity/v3/users/2fd61952074f49b9bed58536e54de503 => generated 275 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:00.790384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9042f5b-5d2f-4873-bb6f-765b935d2708 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:00.791149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9042f5b-5d2f-4873-bb6f-765b935d2708 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:00.791855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9042f5b-5d2f-4873-bb6f-765b935d2708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:00.823023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e9042f5b-5d2f-4873-bb6f-765b935d2708 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:00.825572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e9042f5b-5d2f-4873-bb6f-765b935d2708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:00.834698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7760818-4aeb-48d4-83d3-3a7b0beac73d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:00.878662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:00.888919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:00.903127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd606a-8e78-4464-a103-37db4a52a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:00.904237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd606a-8e78-4464-a103-37db4a52a63f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:00.904783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd606a-8e78-4464-a103-37db4a52a63f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:00.905292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd606a-8e78-4464-a103-37db4a52a63f None admin] PATH_INFO: `/v3/projects/16188004a08942a0a5352ad3d555aff7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:00.907491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd606a-8e78-4464-a103-37db4a52a63f None admin] RBAC: Authorizing `identity:get_project(project_id=16188004a08942a0a5352ad3d555aff7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:00.910484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd606a-8e78-4464-a103-37db4a52a63f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:00.913745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 88/177] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:00 2022] GET /identity/v3/projects/16188004a08942a0a5352ad3d555aff7 => generated 312 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:00.916268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a49423-636e-4b66-b9a9-c73e57e4dffc None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:00.916810 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a49423-636e-4b66-b9a9-c73e57e4dffc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:00.917305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a49423-636e-4b66-b9a9-c73e57e4dffc None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:00.920259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 89/178] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:00.921814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a49423-636e-4b66-b9a9-c73e57e4dffc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:00.964817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 90/179] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:00.972639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9042f5b-5d2f-4873-bb6f-765b935d2708 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:01.125489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a49423-636e-4b66-b9a9-c73e57e4dffc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:01.181069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9042f5b-5d2f-4873-bb6f-765b935d2708 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:01.186442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:01.187403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:01.187866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:01.188356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:01.246603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ee505-4146-4742-9dd5-6e39c99e74fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:01.248104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ee505-4146-4742-9dd5-6e39c99e74fc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:01.248878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ee505-4146-4742-9dd5-6e39c99e74fc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:01.249649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ee505-4146-4742-9dd5-6e39c99e74fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:01.251194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ee505-4146-4742-9dd5-6e39c99e74fc None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=16188004a08942a0a5352ad3d555aff7, user.id=2fd61952074f49b9bed58536e54de503) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:01.251964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ee505-4146-4742-9dd5-6e39c99e74fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:01.254859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ee505-4146-4742-9dd5-6e39c99e74fc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:01.266824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 91/180] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:00 2022] GET /identity/v3/role_assignments?user.id=2fd61952074f49b9bed58536e54de503&scope.project.id=16188004a08942a0a5352ad3d555aff7&role.id=5c592b5b170d48b0825640b8283425de => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:01.269556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c8a5-f8b1-4a4d-9024-4766dbc1535b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:01.270450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c8a5-f8b1-4a4d-9024-4766dbc1535b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:01.271254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659c8a5-f8b1-4a4d-9024-4766dbc1535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:01.303312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6659c8a5-f8b1-4a4d-9024-4766dbc1535b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:01.306138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6659c8a5-f8b1-4a4d-9024-4766dbc1535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:01.308268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:01.321625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:01.324448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:01.354613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:01.445595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 92/181] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:01.448136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eaa94-a33a-4f19-8511-18365d91c30e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:01.449035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eaa94-a33a-4f19-8511-18365d91c30e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:01.449804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eaa94-a33a-4f19-8511-18365d91c30e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:01.453131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 93/182] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:01.488640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505244-7cca-46ba-8a9d-c46635be9f0e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:01.489584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505244-7cca-46ba-8a9d-c46635be9f0e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:01.490469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505244-7cca-46ba-8a9d-c46635be9f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:01.523259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-30505244-7cca-46ba-8a9d-c46635be9f0e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:01.525828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-30505244-7cca-46ba-8a9d-c46635be9f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:01.594243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 90/183] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:01.595996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:01.651763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 94/184] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:01.653733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30505244-7cca-46ba-8a9d-c46635be9f0e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:01.816510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:01.829131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95fb4f-cc64-4de4-8155-2895b3faff65 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:01.904402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125e29e-e179-4ea1-aa8d-b726da837e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:01.905857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125e29e-e179-4ea1-aa8d-b726da837e05 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:01.906486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125e29e-e179-4ea1-aa8d-b726da837e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:01.907149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125e29e-e179-4ea1-aa8d-b726da837e05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:01.927104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125e29e-e179-4ea1-aa8d-b726da837e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:01.930543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d125e29e-e179-4ea1-aa8d-b726da837e05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:01.940144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:01.942152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:01.944333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 91/185] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:47:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:01.948482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bdc3-eb7a-4acf-bf6c-4490b96621df None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:01.948482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bdc3-eb7a-4acf-bf6c-4490b96621df None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:01.948482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bdc3-eb7a-4acf-bf6c-4490b96621df None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:01.950824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 92/186] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:01.957961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135bdc3-eb7a-4acf-bf6c-4490b96621df None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:01.964884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30505244-7cca-46ba-8a9d-c46635be9f0e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.031455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c16a37-3d19-4358-894b-115c536d67ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.032522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c16a37-3d19-4358-894b-115c536d67ca None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.032972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c16a37-3d19-4358-894b-115c536d67ca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.033401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c16a37-3d19-4358-894b-115c536d67ca None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.035333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c16a37-3d19-4358-894b-115c536d67ca None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:02.038048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c16a37-3d19-4358-894b-115c536d67ca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:02.041195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 95/187] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:01 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:02.043186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef78ea-043a-452e-a603-7673a46122c9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.043644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef78ea-043a-452e-a603-7673a46122c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.044082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef78ea-043a-452e-a603-7673a46122c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.097599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-14ef78ea-043a-452e-a603-7673a46122c9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:02.099758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14ef78ea-043a-452e-a603-7673a46122c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:02.142502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135bdc3-eb7a-4acf-bf6c-4490b96621df None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.199049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.200004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.200451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.200894 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.201902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:02.202433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:02.204137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:02.223853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 93/188] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:02.225765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:02.235925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 96/189] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:02.236520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef78ea-043a-452e-a603-7673a46122c9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:02.253747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:02.374671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:02.406690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef78ea-043a-452e-a603-7673a46122c9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.410473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d969b-39ee-4b49-a743-a7aecf245fdb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.470584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.470857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.471534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.471712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.471979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.472114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.472407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.472579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.473746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] RBAC: Adding query filter params (role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=2fd61952074f49b9bed58536e54de503) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:02.474241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:02.474563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:02.476926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:02.477491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:02.484521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 97/190] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:02.484521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:02.496581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 94/191] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:02 2022] GET /identity/v3/role_assignments?user.id=2fd61952074f49b9bed58536e54de503&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:02.497171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:02.649623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:02.690081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394ca6e-f5c4-40f0-9468-6785c2b340ce None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.698355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8128d-9de8-4558-a5fe-ca585e10d136 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.751275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.752362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.753440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.753812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.754232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.754518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.754827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] PATH_INFO: `/v3/users/aeb22305b70a4d3f93d6007c0a2ab090` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.756474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.774767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:02.776412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb22305b70a4d3f93d6007c0a2ab090)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:02.778367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:02.779967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:02.780901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:02.781693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:02.782322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 95/192] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:02 2022] GET /identity/v3/users/aeb22305b70a4d3f93d6007c0a2ab090 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:02.782322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:02.782945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 98/193] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:02.788907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:02.819117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:02.901747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb823708-1815-460b-aab6-1298f0b8fe81 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.938700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19ea4-6cae-4470-a56d-8a88000152d7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:02.944076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.945091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.945583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.946061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:02.947085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:02.949882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:02.962496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:02.976752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 96/194] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:02 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:02.977632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:02.982977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:02.984021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:02.984513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:02.984976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.008622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:03.017300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:03.019438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 99/195] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:03.021336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:03.111949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d5c1d-663f-422d-a24b-0ce78f60a611 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.147272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:03.156142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.156958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.157099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.157420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.158680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:03.159270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:03.160734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:03.178510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 97/196] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:03.178510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:03.210758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d71982-195f-43e4-8d97-1163da871c17 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.257101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:03.260880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.261875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.262136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.262760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.339701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0cfd-37b5-451a-9b4b-199d82eb7ad1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.362915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:03.365803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:03.415652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.416193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.416508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.416776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.417460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:03.417758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:03.419216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:03.430407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 98/197] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:03.430407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:03.479996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:03.516196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 100/198] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:03.517157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:03.586666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4a9b3-dcc6-4016-8213-7d450261014b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.609759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:03.627876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.628968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.629494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.629761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd647067-f03c-4b0d-8c88-72bcb271a33c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.630191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] PATH_INFO: `/v3/users/aeb22305b70a4d3f93d6007c0a2ab090` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.632206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb22305b70a4d3f93d6007c0a2ab090)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:03.634067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:03.636356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 99/199] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:03 2022] GET /identity/v3/users/aeb22305b70a4d3f93d6007c0a2ab090 => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:03.637785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:03.674575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.675293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.675466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.675694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] PATH_INFO: `/v3/users/f13c21a275964533844c82cd020a6752` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.676968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] RBAC: Authorizing `identity:get_user(user_id=f13c21a275964533844c82cd020a6752)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:03.678528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:03.680610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 101/200] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:03 2022] GET /identity/v3/users/f13c21a275964533844c82cd020a6752 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:03.681407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:03.735230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:03.758003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f3552-e566-4879-8647-f07db0e4a5a3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.795551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.796619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.797113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.797571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.799061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:03.800850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:03.802921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 100/201] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:03 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:03.804245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:03.804736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378fe34-e7ba-4c99-86f3-97b2e7e5df0c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.844747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.845384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.845665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.845936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.868727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:03.919949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf738-beac-46e9-84f6-6a186eec8e7f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:03.929085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:03.931002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:03.962524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:03.963539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:03.963725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:03.964092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:03.983478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:04.044750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:04.046331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:04.064349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 102/202] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:04.070307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:04.111303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:04.161447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 101/203] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:04.162375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:04.193604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd024f-2b85-4cbc-9dad-e992203b9d14 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:04.233446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:04.234404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:04.234829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:04.235391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:04.236295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=aeb22305b70a4d3f93d6007c0a2ab090) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:04.236814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:04.238606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:04.245420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 103/204] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:04 2022] GET /identity/v3/role_assignments?user.id=aeb22305b70a4d3f93d6007c0a2ab090&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:04.246532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:04.250681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:04.315928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078779-112f-4d86-a45b-23be3fc66958 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:04.385219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753c208-872c-405b-a443-4de4b000845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:04.386422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753c208-872c-405b-a443-4de4b000845c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:04.387934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753c208-872c-405b-a443-4de4b000845c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:04.388694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753c208-872c-405b-a443-4de4b000845c None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:04.395523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fe993-5675-4fe6-adba-4f5983546150 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:04.399173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753c208-872c-405b-a443-4de4b000845c None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:04.402062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753c208-872c-405b-a443-4de4b000845c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:04.404862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 102/205] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:04 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:04.428462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753c208-872c-405b-a443-4de4b000845c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:04.480839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:04.481977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:04.482505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:04.483109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:04.484755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:04.486602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:04.488964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 104/206] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:04 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:04.498334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:04.505281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:04.656665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753c208-872c-405b-a443-4de4b000845c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:04.657455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:04.726691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477bd7f-c269-4bc9-be25-178803d18aec None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:04.753868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5e96f-9dab-42e4-9a36-6ee885655f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:04.754537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5e96f-9dab-42e4-9a36-6ee885655f56 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:04.754943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5e96f-9dab-42e4-9a36-6ee885655f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:04.755212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5e96f-9dab-42e4-9a36-6ee885655f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:04.755966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5e96f-9dab-42e4-9a36-6ee885655f56 None admin] RBAC: Adding query filter params (role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=aeb22305b70a4d3f93d6007c0a2ab090) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:04.756276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5e96f-9dab-42e4-9a36-6ee885655f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:04.757868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5e96f-9dab-42e4-9a36-6ee885655f56 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:04.777733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 103/207] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:04 2022] GET /identity/v3/role_assignments?user.id=aeb22305b70a4d3f93d6007c0a2ab090&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:04.798149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe7c27-12a2-4259-8df1-10ddeba71c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:04.799528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe7c27-12a2-4259-8df1-10ddeba71c12 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:04.799991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe7c27-12a2-4259-8df1-10ddeba71c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:04.800403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe7c27-12a2-4259-8df1-10ddeba71c12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:04.801501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe7c27-12a2-4259-8df1-10ddeba71c12 None admin] RBAC: Adding query filter params (role.id=89cf360b7f5f43ec85bcf8dea35985dd, scope.project.id=a5a5bade20834645af2806f86402c383, user.id=f13c21a275964533844c82cd020a6752) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:04.802214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe7c27-12a2-4259-8df1-10ddeba71c12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:04.804712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe7c27-12a2-4259-8df1-10ddeba71c12 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:04.818271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 105/208] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:04 2022] GET /identity/v3/role_assignments?user.id=f13c21a275964533844c82cd020a6752&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=89cf360b7f5f43ec85bcf8dea35985dd => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:06.018625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee272327-1f8a-4fd4-ba17-9498c4f59bb4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.019141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee272327-1f8a-4fd4-ba17-9498c4f59bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.019141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee272327-1f8a-4fd4-ba17-9498c4f59bb4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.020626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 104/209] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:06.027244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a594e-f592-46ec-8c7f-b79114d0ae4e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.027788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a594e-f592-46ec-8c7f-b79114d0ae4e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.027995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a594e-f592-46ec-8c7f-b79114d0ae4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.049233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8b4a594e-f592-46ec-8c7f-b79114d0ae4e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:06.050517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8b4a594e-f592-46ec-8c7f-b79114d0ae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:06.128266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 106/210] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:06.162721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1be89-74e0-4158-af6d-93d03ecea9e9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.163431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1be89-74e0-4158-af6d-93d03ecea9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.163900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1be89-74e0-4158-af6d-93d03ecea9e9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.165852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 105/211] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:06.171903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901d7b8-5120-41af-94b4-555a3938c612 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.172537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901d7b8-5120-41af-94b4-555a3938c612 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.173020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901d7b8-5120-41af-94b4-555a3938c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.193215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2901d7b8-5120-41af-94b4-555a3938c612 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:06.195046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2901d7b8-5120-41af-94b4-555a3938c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:06.266334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:06.275120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 107/212] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:06.547524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1be89-74e0-4158-af6d-93d03ecea9e9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:06.639862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901d7b8-5120-41af-94b4-555a3938c612 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:06.696238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:06.722193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1be89-74e0-4158-af6d-93d03ecea9e9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:06.785841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae2886-ac0c-49a0-9eaf-cc39987668ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:06.786508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae2886-ac0c-49a0-9eaf-cc39987668ed None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.786785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae2886-ac0c-49a0-9eaf-cc39987668ed None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.787052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae2886-ac0c-49a0-9eaf-cc39987668ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.809223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae2886-ac0c-49a0-9eaf-cc39987668ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-bcae2886-ac0c-49a0-9eaf-cc39987668ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:06.820141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:06.821983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:06.823581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 106/213] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:06.824920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82d94-ff69-443a-854d-bbb397795412 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.825044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82d94-ff69-443a-854d-bbb397795412 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.826021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a82d94-ff69-443a-854d-bbb397795412 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.828079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 107/214] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:06.829480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a82d94-ff69-443a-854d-bbb397795412 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:06.840890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901d7b8-5120-41af-94b4-555a3938c612 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:06.890333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39167ee6-c814-4a95-9f0e-5496fe300867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:06.891510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167ee6-c814-4a95-9f0e-5496fe300867 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.891989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167ee6-c814-4a95-9f0e-5496fe300867 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.892478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167ee6-c814-4a95-9f0e-5496fe300867 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.894597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39167ee6-c814-4a95-9f0e-5496fe300867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:06.897498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39167ee6-c814-4a95-9f0e-5496fe300867 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:06.900728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 108/215] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:06.908255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da8b45-a594-4956-89dd-048d08b33dba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:06.909030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da8b45-a594-4956-89dd-048d08b33dba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:06.909872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da8b45-a594-4956-89dd-048d08b33dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:06.942039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c8da8b45-a594-4956-89dd-048d08b33dba None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:06.944051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c8da8b45-a594-4956-89dd-048d08b33dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:06.991607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a82d94-ff69-443a-854d-bbb397795412 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:07.048963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:07.050402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:07.050402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:07.050852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:07.051881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:07.052419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:07.055083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:07.057922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:07.069725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 108/216] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:07.071265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:07.092465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 109/217] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:07.092465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da8b45-a594-4956-89dd-048d08b33dba None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:07.101819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:07.290719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da8b45-a594-4956-89dd-048d08b33dba None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:07.295077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f3e8b-955f-4818-b471-abae93e9ce1f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:07.341390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:07.342294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:07.342580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:07.342853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] PATH_INFO: `/v3/users/95c02cc992744d84a2266fc7f0dfd329` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:07.365505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e05fe-9be5-4423-b9f3-088c6d4f656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:07.366340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e05fe-9be5-4423-b9f3-088c6d4f656d None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:07.366623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e05fe-9be5-4423-b9f3-088c6d4f656d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:07.367031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e05fe-9be5-4423-b9f3-088c6d4f656d None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:07.367736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e05fe-9be5-4423-b9f3-088c6d4f656d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:07.368103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] RBAC: Authorizing `identity:get_user(user_id=95c02cc992744d84a2266fc7f0dfd329)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:07.369366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e05fe-9be5-4423-b9f3-088c6d4f656d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:07.371307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:07.374819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 110/218] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:07 2022] GET /identity/v3/users/95c02cc992744d84a2266fc7f0dfd329 => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:07.380791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:07.417111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 109/219] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:07 2022] POST /identity/v3/users => generated 297 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:07.446847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:07.470578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d24019-e714-49c6-ac45-2a059583c336 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:07.470975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d24019-e714-49c6-ac45-2a059583c336 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:07.471113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d24019-e714-49c6-ac45-2a059583c336 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:07.472615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 110/220] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:07.478635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1678-b49f-4850-9df0-78a6afaf203b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:07.479031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1678-b49f-4850-9df0-78a6afaf203b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:07.479412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1678-b49f-4850-9df0-78a6afaf203b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:07.501923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6dfb1678-b49f-4850-9df0-78a6afaf203b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:07.503281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6dfb1678-b49f-4850-9df0-78a6afaf203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:07.546870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3384ba-e168-43ef-8464-650127b3359f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:07.583361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:07.600568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 111/221] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:07.604233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb1678-b49f-4850-9df0-78a6afaf203b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:07.622503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:07.624016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:07.624840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:07.625541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:07.763995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:07.766775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:07.793781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb1678-b49f-4850-9df0-78a6afaf203b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:07.854682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:07.855749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:07.856227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:07.856736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:07.868495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:07.875306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:07.881088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:07.882941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:07.884476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 112/222] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:47:07.893584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:07.974906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 111/223] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 17:47:07.975896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:07.990831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:08.061691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af8552-5313-4d54-93f9-617af6733070 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.125220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.126733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.127497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.127757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68266854-cea7-40b4-8a80-381b5136f139 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.129263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.131164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:08.132161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:08.135220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:08.153650 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 113/224] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:08.155745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.168951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.169602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.169880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.170152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.182199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:08.188049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:08.189790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 112/225] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:08.191359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.209277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:08.318396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd4af4-b322-43f5-b61b-84fed0a961a3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.363959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157110c5-6a21-4910-9c36-cc84d1391e83 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.383223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:08.384725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.385601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.387221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.388316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] PATH_INFO: `/v3/users/aeb22305b70a4d3f93d6007c0a2ab090` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.390099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] RBAC: Authorizing `identity:get_user(user_id=aeb22305b70a4d3f93d6007c0a2ab090)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:08.392530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:08.394560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 113/226] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:08 2022] GET /identity/v3/users/aeb22305b70a4d3f93d6007c0a2ab090 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:08.395872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.403414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.404220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.404624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.404996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.406404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:08.408003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:08.410046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 114/227] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:08 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:08.411001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.495761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:08.522517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a21f53-3fc7-4009-af58-d99692cc20ba None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.562250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.563031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.563431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.563802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.564617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:08.565025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:08.566522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:08.576440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 114/228] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:08.583291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.583483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bdcb6-d07d-4713-b423-32e1482870dc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.620081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.620950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.621329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.621699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.622964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:08.624618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:08.626650 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 115/229] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:08 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:08.627404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.694928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:08.750552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ab402-71ca-4016-b1f5-9bb6549416e4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.762208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c66d7-2c01-4e40-b73b-bd231317fe3f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:08.814789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.815799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.816318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.816756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] PATH_INFO: `/v3/users/aeb22305b70a4d3f93d6007c0a2ab090` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.818173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb22305b70a4d3f93d6007c0a2ab090)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:08.818368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:08.819720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:08.820305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:08.820419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:08.821065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:08.822415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 116/230] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:08 2022] GET /identity/v3/users/aeb22305b70a4d3f93d6007c0a2ab090 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:08.822551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=95c02cc992744d84a2266fc7f0dfd329) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:08.823200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:08.823616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.826477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:08.838234 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 115/231] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:08 2022] GET /identity/v3/role_assignments?user.id=95c02cc992744d84a2266fc7f0dfd329&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=5c592b5b170d48b0825640b8283425de => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:08.839695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:08.880305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:09.021254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bddd9-a1ff-4cb5-968f-1dcab9c91732 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:09.077718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:09.105498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:09.106977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:09.107641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:09.108288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/aeb22305b70a4d3f93d6007c0a2ab090/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:09.112070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] RBAC: Authorizing `identity:create_grant(project_id=16f3442e76ed418d936b9a5eb6f19e81, user_id=aeb22305b70a4d3f93d6007c0a2ab090, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:09.114990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:09.139617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 117/232] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:47:08 2022] PUT /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/aeb22305b70a4d3f93d6007c0a2ab090/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:09.182295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0d5e6-0fc6-4ab0-9cd7-44f49f2dc164 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:09.263636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeada972-7e99-44a5-917b-fcc26ae4df57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:09.265328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeada972-7e99-44a5-917b-fcc26ae4df57 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:09.265979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeada972-7e99-44a5-917b-fcc26ae4df57 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:09.266627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeada972-7e99-44a5-917b-fcc26ae4df57 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:09.268794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeada972-7e99-44a5-917b-fcc26ae4df57 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:09.272136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeada972-7e99-44a5-917b-fcc26ae4df57 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:09.275475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 116/233] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:08 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:09.282286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:09.475722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238fba5-1606-4f85-92ed-465e02d36505 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:09.531558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0548ef-9476-4596-8a48-887e40d7cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:09.532167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0548ef-9476-4596-8a48-887e40d7cf98 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:09.532564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0548ef-9476-4596-8a48-887e40d7cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:09.532752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0548ef-9476-4596-8a48-887e40d7cf98 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/aeb22305b70a4d3f93d6007c0a2ab090/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:09.538042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0548ef-9476-4596-8a48-887e40d7cf98 None admin] RBAC: Authorizing `identity:create_grant(project_id=16f3442e76ed418d936b9a5eb6f19e81, user_id=aeb22305b70a4d3f93d6007c0a2ab090, role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:09.539580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0548ef-9476-4596-8a48-887e40d7cf98 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:09.562665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 118/234] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:47:09 2022] PUT /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/aeb22305b70a4d3f93d6007c0a2ab090/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 17:47:10.923142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239016e-523f-4957-93ca-77d5de0afd50 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:10.923666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239016e-523f-4957-93ca-77d5de0afd50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:10.924072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239016e-523f-4957-93ca-77d5de0afd50 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:10.926750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 117/235] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:10.933764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a940b3-6dba-466b-b087-6e1b56d3356d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:10.934266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a940b3-6dba-466b-b087-6e1b56d3356d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:10.934707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a940b3-6dba-466b-b087-6e1b56d3356d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:10.976700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-40a940b3-6dba-466b-b087-6e1b56d3356d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:10.978761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-40a940b3-6dba-466b-b087-6e1b56d3356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:11.063245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:11.139831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 119/236] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:11.331200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78be33-469e-4948-aa2d-e697a593047c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:11.331590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78be33-469e-4948-aa2d-e697a593047c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:11.332052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78be33-469e-4948-aa2d-e697a593047c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:11.334786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 118/237] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:11.343044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aefadc-18ba-4f18-ab66-38ae93aa6875 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:11.343523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aefadc-18ba-4f18-ab66-38ae93aa6875 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:11.343968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aefadc-18ba-4f18-ab66-38ae93aa6875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:11.380893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-59aefadc-18ba-4f18-ab66-38ae93aa6875 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:11.382948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-59aefadc-18ba-4f18-ab66-38ae93aa6875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:11.513490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 120/238] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:11.658374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78be33-469e-4948-aa2d-e697a593047c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:11.818649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:11.859910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78be33-469e-4948-aa2d-e697a593047c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:11.917427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:11.918016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:11.918301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:11.918576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:11.938681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:11.943768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:11.945498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:11.946941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 119/239] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:47:11.952010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aefadc-18ba-4f18-ab66-38ae93aa6875 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:12.074328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:12.227522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aefadc-18ba-4f18-ab66-38ae93aa6875 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:12.229529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:12.271201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d1ba1-8e76-4d56-8bc4-cfbd67975f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:12.271825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d1ba1-8e76-4d56-8bc4-cfbd67975f07 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.272169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d1ba1-8e76-4d56-8bc4-cfbd67975f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.272464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d1ba1-8e76-4d56-8bc4-cfbd67975f07 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.273151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d1ba1-8e76-4d56-8bc4-cfbd67975f07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:12.273448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d1ba1-8e76-4d56-8bc4-cfbd67975f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:12.274892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d1ba1-8e76-4d56-8bc4-cfbd67975f07 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:12.285095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 121/240] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:12.286485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5c7f-2afc-4503-be4d-6bca1ad0af4b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.286781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5c7f-2afc-4503-be4d-6bca1ad0af4b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.287046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5c7f-2afc-4503-be4d-6bca1ad0af4b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.288352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 122/241] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:12.293638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef5c7f-2afc-4503-be4d-6bca1ad0af4b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:12.378224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04de25-83fc-4787-a88b-f2bb8e73774e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:12.433759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef5c7f-2afc-4503-be4d-6bca1ad0af4b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:12.444527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169030ad-6384-4f90-ae89-a8f3ea3fc2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:12.445345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169030ad-6384-4f90-ae89-a8f3ea3fc2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.445716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169030ad-6384-4f90-ae89-a8f3ea3fc2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.446101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169030ad-6384-4f90-ae89-a8f3ea3fc2d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.447379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169030ad-6384-4f90-ae89-a8f3ea3fc2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:12.448998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169030ad-6384-4f90-ae89-a8f3ea3fc2d8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:12.451202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 120/242] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:12.452080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc46f64-8a2e-41eb-8678-0fd5754d210b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.452580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc46f64-8a2e-41eb-8678-0fd5754d210b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.452944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc46f64-8a2e-41eb-8678-0fd5754d210b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.460606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:12.475857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ccc46f64-8a2e-41eb-8678-0fd5754d210b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:12.477302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ccc46f64-8a2e-41eb-8678-0fd5754d210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:12.507479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:12.508972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.509654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.510331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] PATH_INFO: `/v3/users/7720c048489d47349e2fed6f452ced0e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.537954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] RBAC: Authorizing `identity:get_user(user_id=7720c048489d47349e2fed6f452ced0e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:12.541405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:12.545370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 123/243] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:12 2022] GET /identity/v3/users/7720c048489d47349e2fed6f452ced0e => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:12.547052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:12.562337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 121/244] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:12.564554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f49c1-f039-4530-89bc-2cc70acf6dc2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.565098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f49c1-f039-4530-89bc-2cc70acf6dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.565569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f49c1-f039-4530-89bc-2cc70acf6dc2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.567688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 122/245] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:12.568766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f49c1-f039-4530-89bc-2cc70acf6dc2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:12.594652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:12.699826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8a5d-a6a6-4943-a5f9-1139712a9344 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:12.743718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d017f-2b49-45a6-81f1-dcf51f403940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:12.744361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d017f-2b49-45a6-81f1-dcf51f403940 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.744671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d017f-2b49-45a6-81f1-dcf51f403940 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.744948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d017f-2b49-45a6-81f1-dcf51f403940 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.745945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d017f-2b49-45a6-81f1-dcf51f403940 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:12.748335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d017f-2b49-45a6-81f1-dcf51f403940 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:12.751196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f49c1-f039-4530-89bc-2cc70acf6dc2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:12.790416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 124/246] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:12 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:12.793168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:12.795499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fea71-794f-460f-9bff-a51c5511ca9b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.795929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fea71-794f-460f-9bff-a51c5511ca9b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.796346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fea71-794f-460f-9bff-a51c5511ca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.799876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:12.801314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:12.801968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:12.802509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:12.833858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1a8fea71-794f-460f-9bff-a51c5511ca9b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:12.835746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1a8fea71-794f-460f-9bff-a51c5511ca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:12.946600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:12.948240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:12.985820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 125/247] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:12.987247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fea71-794f-460f-9bff-a51c5511ca9b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:13.010412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:13.154149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 123/248] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:13.159970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:13.194863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:13.203509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fea71-794f-460f-9bff-a51c5511ca9b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:13.267455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d06a-d627-4422-aeab-5842e9b5f8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:13.268754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d06a-d627-4422-aeab-5842e9b5f8af None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:13.269299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d06a-d627-4422-aeab-5842e9b5f8af None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:13.269765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d06a-d627-4422-aeab-5842e9b5f8af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:13.275274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30b5a5-5222-4e33-a24d-9c543b6e1b5f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:13.284666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d06a-d627-4422-aeab-5842e9b5f8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2675d06a-d627-4422-aeab-5842e9b5f8af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:13.290134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:13.291712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 126/249] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:13.293198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a57d2-ce83-4f94-84da-f968ca29d8b9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:13.294508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a57d2-ce83-4f94-84da-f968ca29d8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:13.294508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a57d2-ce83-4f94-84da-f968ca29d8b9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:13.295777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 127/250] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:13.300159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a57d2-ce83-4f94-84da-f968ca29d8b9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:13.332189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474a438-c704-4fe8-a2cc-e072ab75d0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:13.332875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474a438-c704-4fe8-a2cc-e072ab75d0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:13.333148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474a438-c704-4fe8-a2cc-e072ab75d0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:13.333413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474a438-c704-4fe8-a2cc-e072ab75d0e6 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:13.334637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474a438-c704-4fe8-a2cc-e072ab75d0e6 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:13.336304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474a438-c704-4fe8-a2cc-e072ab75d0e6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:13.338610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 124/251] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:13 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:13.339842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae2b9-93f9-43aa-810d-3b9b7a776856 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:13.340112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae2b9-93f9-43aa-810d-3b9b7a776856 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:13.340373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eae2b9-93f9-43aa-810d-3b9b7a776856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:13.365403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-45eae2b9-93f9-43aa-810d-3b9b7a776856 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:13.366876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-45eae2b9-93f9-43aa-810d-3b9b7a776856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:13.422630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:13.473559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a57d2-ce83-4f94-84da-f968ca29d8b9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:13.475920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 125/252] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:13.479340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae2b9-93f9-43aa-810d-3b9b7a776856 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:13.516610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:13.532993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:13.533795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:13.534175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:13.534568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:13.535329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:13.536498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:13.538090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:13.551790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 128/253] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:13.556515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:13.675599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eae2b9-93f9-43aa-810d-3b9b7a776856 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:13.695120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b27b-a526-4c5f-95d4-51101aa5325c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:13.732505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:13.754009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:13.754835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:13.755222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:13.755529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:13.755908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:13.757276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:13.757940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:13.758601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:13.760130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] RBAC: Adding query filter params (role.id=89cf360b7f5f43ec85bcf8dea35985dd, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=7720c048489d47349e2fed6f452ced0e) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:13.760893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:13.764711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:13.776645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 126/254] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:13 2022] GET /identity/v3/role_assignments?user.id=7720c048489d47349e2fed6f452ced0e&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=89cf360b7f5f43ec85bcf8dea35985dd => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:13.778592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:13.779089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:13.796996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:13.798982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:13.800813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:13.802013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 129/255] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:13.806822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:13.996965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:14.009667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96454128-1b4b-43cc-a0ce-01d63fed139f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:14.192457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:14.192457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:14.192457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:14.192994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] PATH_INFO: `/v3/users/95c02cc992744d84a2266fc7f0dfd329` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:14.196458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] RBAC: Authorizing `identity:get_user(user_id=95c02cc992744d84a2266fc7f0dfd329)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:14.196458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:14.204828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 127/256] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:13 2022] GET /identity/v3/users/95c02cc992744d84a2266fc7f0dfd329 => generated 297 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:14.206676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:14.232460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecdbed-b3db-4e99-8844-43a108655be7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:14.382109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:14.383010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:14.383415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:14.383811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:14.391852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:14.392317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:14.394630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:14.416251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 130/257] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 610 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:14.417298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:14.583264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64394510-d44b-4bdc-b1e3-592fd2334655 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:14.633879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:14.723758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:14.724952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:14.725525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:14.726005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:14.767664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:14.791179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:14.792887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:14.794380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:14.795661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 128/258] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 590 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:14.806885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:14.858669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:14.864217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a10aa-cf4a-41a7-9094-d5c5bf7f49fa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:14.976394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:14.977387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:14.977868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:14.978344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:14.984747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:14.992015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:14.995304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 131/259] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:14 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 578 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:15.005250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:15.105160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2a4dc-972d-4da3-881e-11a3c83eda67 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:15.175945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:15.177062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:15.177534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:15.178007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] PATH_INFO: `/v3/users/aeb22305b70a4d3f93d6007c0a2ab090` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:15.183052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] RBAC: Authorizing `identity:get_user(user_id=aeb22305b70a4d3f93d6007c0a2ab090)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:15.185808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:15.193342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 129/260] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:14 2022] GET /identity/v3/users/aeb22305b70a4d3f93d6007c0a2ab090 => generated 283 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:15.194683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:15.314601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f465ed8-4e67-491d-86a0-83b3a44641a0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:15.365244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:15.409429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:15.410082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:15.410466 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:15.410748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:15.411473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:15.411777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:15.413376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:15.421594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:15.437634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 132/261] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:15.439371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:15.440203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb267db-a2b6-4de2-a370-aa3ddf87e73d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:15.508120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:15.508872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:15.509154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:15.510811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/95c02cc992744d84a2266fc7f0dfd329/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:15.516350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] RBAC: Authorizing `identity:create_grant(project_id=16f3442e76ed418d936b9a5eb6f19e81, user_id=95c02cc992744d84a2266fc7f0dfd329, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:15.517935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:15.559557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 130/262] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:47:15 2022] PUT /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/95c02cc992744d84a2266fc7f0dfd329/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:15.561353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:15.620865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c23aa0-d83c-4d4a-ac0f-ba94f838c44c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:15.687478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:15.688165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:15.688463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:15.688732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:15.809569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:15.841552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:15.843184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:15.979733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55d4b3-1c91-4331-902b-2b4cc7c6df63 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:15.999130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 133/263] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:16.028826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:16.070850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:16.071895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:16.072375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:16.072877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] PATH_INFO: `/v3/users/aeb22305b70a4d3f93d6007c0a2ab090` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:16.074839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] RBAC: Authorizing `identity:get_user(user_id=aeb22305b70a4d3f93d6007c0a2ab090)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:16.077536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:16.080448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 131/264] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:15 2022] GET /identity/v3/users/aeb22305b70a4d3f93d6007c0a2ab090 => generated 283 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:16.089293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:16.140534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:16.151060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:16.176915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa21daa-5f9e-44d9-9689-1d9bc40dfea8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:16.275437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9c354-02de-4a75-ae2e-b6bd2090ffca None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:16.281743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:16.282940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:16.283424 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:16.283915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:16.286088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:16.289427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:16.292628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 134/265] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:16 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:16.302140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:16.314060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:16.314704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:16.314978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:16.315263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:16.395763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:16.397385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:16.419414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:16.517697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 132/266] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 17:47:16.525313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:16.591255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0d1d7-f631-48b7-9da9-62d8861fe104 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:16.654931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:16.655962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:16.656468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:16.656954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:16.657200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc47e7-36eb-4b51-baa1-e46953bef678 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:16.658334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=aeb22305b70a4d3f93d6007c0a2ab090) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:16.658869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:16.661784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:16.673461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 135/267] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:16 2022] GET /identity/v3/role_assignments?user.id=aeb22305b70a4d3f93d6007c0a2ab090&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:16.677436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:16.704823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f85ee-ad3f-4974-a92d-ed7171cce395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:16.705549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f85ee-ad3f-4974-a92d-ed7171cce395 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:16.705832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f85ee-ad3f-4974-a92d-ed7171cce395 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:16.706102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f85ee-ad3f-4974-a92d-ed7171cce395 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:16.707307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f85ee-ad3f-4974-a92d-ed7171cce395 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:16.708860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f85ee-ad3f-4974-a92d-ed7171cce395 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:16.711069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 133/268] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:16 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:16.720328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:16.764747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:16.943817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b3239-cc18-429a-a03a-1814a6f7e6bb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:17.021765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918968e4-1b03-42f6-a609-7e6b7dbff72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:17.023067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918968e4-1b03-42f6-a609-7e6b7dbff72a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:17.023562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918968e4-1b03-42f6-a609-7e6b7dbff72a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:17.024030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918968e4-1b03-42f6-a609-7e6b7dbff72a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:17.025500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918968e4-1b03-42f6-a609-7e6b7dbff72a None admin] RBAC: Adding query filter params (role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=aeb22305b70a4d3f93d6007c0a2ab090) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:17.026044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918968e4-1b03-42f6-a609-7e6b7dbff72a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:17.028917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918968e4-1b03-42f6-a609-7e6b7dbff72a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:17.042358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 136/269] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:16 2022] GET /identity/v3/role_assignments?user.id=aeb22305b70a4d3f93d6007c0a2ab090&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:17.157444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694df8-bade-470b-8aa9-4922a9523f38 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:17.158487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694df8-bade-470b-8aa9-4922a9523f38 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:17.158955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694df8-bade-470b-8aa9-4922a9523f38 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:17.160981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 134/270] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:17.183428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eba2c-8b09-4330-b53f-ddf87e949bf9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:17.183884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eba2c-8b09-4330-b53f-ddf87e949bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:17.184341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eba2c-8b09-4330-b53f-ddf87e949bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:17.225844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b42eba2c-8b09-4330-b53f-ddf87e949bf9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:17.226989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b42eba2c-8b09-4330-b53f-ddf87e949bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:17.318689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 137/271] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:17.476657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860aed3-29f0-4640-81bb-5d41020c6316 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:17.477343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860aed3-29f0-4640-81bb-5d41020c6316 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:17.477820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860aed3-29f0-4640-81bb-5d41020c6316 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:17.479443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 135/272] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:17.485804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5870da9-6d42-4494-917a-05836dd5affb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:17.486434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5870da9-6d42-4494-917a-05836dd5affb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:17.486900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5870da9-6d42-4494-917a-05836dd5affb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:17.513288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a5870da9-6d42-4494-917a-05836dd5affb None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:17.514919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a5870da9-6d42-4494-917a-05836dd5affb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:17.517549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:17.643621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 138/273] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:17.897679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860aed3-29f0-4640-81bb-5d41020c6316 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:18.005193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5870da9-6d42-4494-917a-05836dd5affb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:18.028722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860aed3-29f0-4640-81bb-5d41020c6316 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:18.074296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:18.075247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:18.075660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:18.076062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:18.090271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:18.095395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:18.096565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:18.097469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:18.099852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 136/274] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:18.111820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:18.208172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5870da9-6d42-4494-917a-05836dd5affb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:18.216341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:18.254296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a4961-19b9-4f50-a9c9-f12acbe1c7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:18.255536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a4961-19b9-4f50-a9c9-f12acbe1c7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:18.255964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a4961-19b9-4f50-a9c9-f12acbe1c7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:18.256520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a4961-19b9-4f50-a9c9-f12acbe1c7a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:18.258590 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a4961-19b9-4f50-a9c9-f12acbe1c7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:18.261920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a4961-19b9-4f50-a9c9-f12acbe1c7a6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:18.265104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 139/275] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:18.267169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ed8b-8d82-4cde-b3d8-fcfcb560c5f6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:18.267592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ed8b-8d82-4cde-b3d8-fcfcb560c5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:18.268018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ed8b-8d82-4cde-b3d8-fcfcb560c5f6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:18.272189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 140/276] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:18.275703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914ed8b-8d82-4cde-b3d8-fcfcb560c5f6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:18.291054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:18.333805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd05e0-05f0-4017-9165-665e62ff486a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:18.393052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651451c5-0d03-411f-b59c-54f112a65fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:18.394146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651451c5-0d03-411f-b59c-54f112a65fbe None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:18.394614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651451c5-0d03-411f-b59c-54f112a65fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:18.395106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651451c5-0d03-411f-b59c-54f112a65fbe None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:18.396358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651451c5-0d03-411f-b59c-54f112a65fbe None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:18.397012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651451c5-0d03-411f-b59c-54f112a65fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:18.399651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651451c5-0d03-411f-b59c-54f112a65fbe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:18.420594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 137/277] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:18.420594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad837552-1ffe-4af0-adb4-acc41d62cc98 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:18.420594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad837552-1ffe-4af0-adb4-acc41d62cc98 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:18.420594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad837552-1ffe-4af0-adb4-acc41d62cc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:18.434280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914ed8b-8d82-4cde-b3d8-fcfcb560c5f6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:18.445087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ad837552-1ffe-4af0-adb4-acc41d62cc98 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:18.447255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ad837552-1ffe-4af0-adb4-acc41d62cc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:18.478019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:18.478949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:18.479349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:18.479757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:18.480866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:18.482520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:18.516576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 141/278] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:18 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:18.516576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:18.579583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 138/279] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:18.640460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:18.696164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed84589-6748-48bb-aff4-12683239bbb7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:18.735497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:18.736165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:18.736467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:18.736742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] PATH_INFO: `/v3/users/34b32c79aff04a32a47c3d4d776ee179` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:18.757379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] RBAC: Authorizing `identity:get_user(user_id=34b32c79aff04a32a47c3d4d776ee179)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:18.758991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:18.761280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 142/280] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:18 2022] GET /identity/v3/users/34b32c79aff04a32a47c3d4d776ee179 => generated 293 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:18.772591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad837552-1ffe-4af0-adb4-acc41d62cc98 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:18.841151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:18.959799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad837552-1ffe-4af0-adb4-acc41d62cc98 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:19.028155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:19.029568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:19.030345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:19.031021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:19.032705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, user.id=34b32c79aff04a32a47c3d4d776ee179, scope.system=all) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:19.033548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:19.036563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:19.052466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 139/281] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 13 17:47:18 2022] GET /identity/v3/role_assignments?user.id=34b32c79aff04a32a47c3d4d776ee179&scope.system=all&role.id=5c592b5b170d48b0825640b8283425de => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:19.061233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:19.191121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c791b-602e-4a00-a261-b2b36c8fa5de None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:19.216951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:19.240759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ef7a-1bd5-4f68-b2f8-66746d2bff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:19.241471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ef7a-1bd5-4f68-b2f8-66746d2bff49 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:19.241749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ef7a-1bd5-4f68-b2f8-66746d2bff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:19.242012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ef7a-1bd5-4f68-b2f8-66746d2bff49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:19.259072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60ef7a-1bd5-4f68-b2f8-66746d2bff49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5f60ef7a-1bd5-4f68-b2f8-66746d2bff49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:19.264701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:19.265687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:19.266473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:19.271802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 143/282] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:19.273648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:19.463375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c548c-1b8c-4694-9220-7ff3899838ce None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:19.463606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c548c-1b8c-4694-9220-7ff3899838ce None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:19.464045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c548c-1b8c-4694-9220-7ff3899838ce None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:19.466212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 144/283] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:19.472045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2e10-9163-4164-ad50-4e9f8e8d262f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:19.472308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2e10-9163-4164-ad50-4e9f8e8d262f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:19.472628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf2e10-9163-4164-ad50-4e9f8e8d262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:19.499840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f45a94-aee2-4995-a3db-85fb7ec7151e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:19.502690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:19.517592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0dcf2e10-9163-4164-ad50-4e9f8e8d262f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:19.518838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0dcf2e10-9163-4164-ad50-4e9f8e8d262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:19.618792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:19.619814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:19.620315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:19.620819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:19.621829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:19.622339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:19.624107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:19.657074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 140/284] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:19.663482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:19.704618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 145/285] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:19.833288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2a6df-c9a3-4594-89c8-3e31009396eb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:19.886767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:19.887799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:19.888075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:19.888349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] PATH_INFO: `/v3/users/7720c048489d47349e2fed6f452ced0e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:19.889650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] RBAC: Authorizing `identity:get_user(user_id=7720c048489d47349e2fed6f452ced0e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:19.891145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:19.893184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 141/286] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:19 2022] GET /identity/v3/users/7720c048489d47349e2fed6f452ced0e => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:19.899131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf2e10-9163-4164-ad50-4e9f8e8d262f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:20.019804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf2e10-9163-4164-ad50-4e9f8e8d262f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:20.032306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:20.073982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:20.074639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:20.074914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:20.075183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:20.077264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:20.078809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:20.080813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 146/287] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:19 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:20.086960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:20.117791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:20.132245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:20.287695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb156198-e57f-48e2-b6fc-a48217a26c8e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:20.332891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400fbb7-bd50-40e7-9ef0-5d467e059afc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:20.344077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:20.351657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:20.351958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:20.352201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/7720c048489d47349e2fed6f452ced0e/roles/89cf360b7f5f43ec85bcf8dea35985dd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:20.354501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] RBAC: Authorizing `identity:create_grant(project_id=16f3442e76ed418d936b9a5eb6f19e81, user_id=7720c048489d47349e2fed6f452ced0e, role_id=89cf360b7f5f43ec85bcf8dea35985dd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:20.358079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:20.386511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 142/288] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:47:20 2022] PUT /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/users/7720c048489d47349e2fed6f452ced0e/roles/89cf360b7f5f43ec85bcf8dea35985dd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:20.424965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:20.425690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:20.426014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:20.426395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:20.439551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:20.445080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:20.447464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:20.449477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 147/289] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:47:20.454467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:20.604399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:20.620615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dfee5-ec6e-40a6-8325-c27702504ae7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:20.667325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:20.667977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:20.668256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:20.668571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:20.669313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:20.669754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:20.672147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:20.696691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 143/290] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:20.705653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:20.766815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:20.853034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27484a29-eba4-4adc-b8a0-f2fac83a526c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:20.900004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:20.900610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:20.900887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:20.901153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] PATH_INFO: `/v3/users/95c02cc992744d84a2266fc7f0dfd329` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:20.905207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] RBAC: Authorizing `identity:get_user(user_id=95c02cc992744d84a2266fc7f0dfd329)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:20.906727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:20.908446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 148/291] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:20 2022] GET /identity/v3/users/95c02cc992744d84a2266fc7f0dfd329 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:20.915411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:21.167461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73855f8b-3e74-435d-91a4-7f41d6554c91 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:21.247889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcd267-724a-4ff0-914d-d3c9eb85ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:21.248921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcd267-724a-4ff0-914d-d3c9eb85ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:21.249411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcd267-724a-4ff0-914d-d3c9eb85ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:21.249908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcd267-724a-4ff0-914d-d3c9eb85ddb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:21.313388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:21.422190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcd267-724a-4ff0-914d-d3c9eb85ddb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:21.424805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcd267-724a-4ff0-914d-d3c9eb85ddb3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:21.665306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 144/292] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 751 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 17:47:21.671451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:21.837854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51694b0e-eca0-42ab-9641-137cc32f5f1b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:21.852002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:21.893669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8974749-68e5-407d-886b-f86d13e1ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:21.894310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8974749-68e5-407d-886b-f86d13e1ecae None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:21.894594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8974749-68e5-407d-886b-f86d13e1ecae None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:21.894867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8974749-68e5-407d-886b-f86d13e1ecae None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:21.896071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8974749-68e5-407d-886b-f86d13e1ecae None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:21.896575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9eac0-4869-4558-85cc-d6c0bb0f899b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:21.897156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9eac0-4869-4558-85cc-d6c0bb0f899b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:21.897620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9eac0-4869-4558-85cc-d6c0bb0f899b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:21.897721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8974749-68e5-407d-886b-f86d13e1ecae None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:21.899377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 145/293] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:21.904383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 149/294] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:21 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:21.905765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9eac0-4869-4558-85cc-d6c0bb0f899b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:21.908101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef109fef-b189-4f54-9775-9795f4517e75 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:21.908656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef109fef-b189-4f54-9775-9795f4517e75 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:21.909130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef109fef-b189-4f54-9775-9795f4517e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:21.961036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ef109fef-b189-4f54-9775-9795f4517e75 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:21.963078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef109fef-b189-4f54-9775-9795f4517e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:22.086116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:22.148122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 150/295] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:22.167276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9eac0-4869-4558-85cc-d6c0bb0f899b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:22.205659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6d51c-7f5d-4606-88c4-e6d543892749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:22.206380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6d51c-7f5d-4606-88c4-e6d543892749 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:22.206651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6d51c-7f5d-4606-88c4-e6d543892749 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:22.206914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6d51c-7f5d-4606-88c4-e6d543892749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:22.207663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6d51c-7f5d-4606-88c4-e6d543892749 None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=95c02cc992744d84a2266fc7f0dfd329) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:22.207970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6d51c-7f5d-4606-88c4-e6d543892749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:22.209497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6d51c-7f5d-4606-88c4-e6d543892749 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:22.216969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 146/296] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:21 2022] GET /identity/v3/role_assignments?user.id=95c02cc992744d84a2266fc7f0dfd329&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=5c592b5b170d48b0825640b8283425de => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:22.242519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c350e-2fb4-4afa-b47d-2489401521f9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:22.242859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c350e-2fb4-4afa-b47d-2489401521f9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:22.243080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c350e-2fb4-4afa-b47d-2489401521f9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:22.244692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 151/297] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:22.264910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42385e4-34f1-43ab-b457-b7e3a1f6bba0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:22.265238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42385e4-34f1-43ab-b457-b7e3a1f6bba0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:22.265538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42385e4-34f1-43ab-b457-b7e3a1f6bba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:22.290789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a42385e4-34f1-43ab-b457-b7e3a1f6bba0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:22.292048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a42385e4-34f1-43ab-b457-b7e3a1f6bba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:22.397585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 147/298] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:22.399101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988de340-1a37-434e-942b-54697832eb64 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:22.399392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988de340-1a37-434e-942b-54697832eb64 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:22.399665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988de340-1a37-434e-942b-54697832eb64 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:22.401034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 152/299] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:22.406833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922099e-46d4-4682-b69f-b500b7ed20c8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:22.407119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922099e-46d4-4682-b69f-b500b7ed20c8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:22.407390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922099e-46d4-4682-b69f-b500b7ed20c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:22.438667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2922099e-46d4-4682-b69f-b500b7ed20c8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:22.439974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2922099e-46d4-4682-b69f-b500b7ed20c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:22.496291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:22.532444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 148/300] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:22.658171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988de340-1a37-434e-942b-54697832eb64 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:22.795138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988de340-1a37-434e-942b-54697832eb64 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:22.798580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:22.870040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:22.871138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:22.871629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:22.872106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:22.874140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922099e-46d4-4682-b69f-b500b7ed20c8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:22.874315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:22.877102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:22.880746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 153/301] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:22.882280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.005548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1285678-0014-4bb1-81e7-912f9fb4eb3e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.026306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922099e-46d4-4682-b69f-b500b7ed20c8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.044802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31700d95-785b-420e-a0fa-458717153631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.045461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31700d95-785b-420e-a0fa-458717153631 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.045743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31700d95-785b-420e-a0fa-458717153631 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.046016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31700d95-785b-420e-a0fa-458717153631 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.058480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31700d95-785b-420e-a0fa-458717153631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-31700d95-785b-420e-a0fa-458717153631 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:23.063447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:23.064375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:23.065238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 154/302] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:23.065919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31700d95-785b-420e-a0fa-458717153631 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.086794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.087745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.088169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.088626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.099551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:23.106385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:23.116400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:23.117377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:23.118184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:23.118882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 149/303] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:23.120310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.183659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31700d95-785b-420e-a0fa-458717153631 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.190756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:23.220530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.221159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.221439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.221719 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] PATH_INFO: `/v3/groups` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.222355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:23.223867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:23.248661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 155/304] 10.222.0.43 () {60 vars in 1134 bytes} [Tue Sep 13 17:47:23 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:23.250184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.290565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139552d2-bd78-4b4e-980c-1d475f025b44 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.343500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.344715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.346204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.346895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.347943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:23.348502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:23.351305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:23.364605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 150/305] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:23.374336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.429849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:23.445408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26bd52-5f82-4a12-aaa2-12db1cff7f4e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.492924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f20fd-c305-4122-b87b-46bcfeba7a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.493759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f20fd-c305-4122-b87b-46bcfeba7a73 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.493913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f20fd-c305-4122-b87b-46bcfeba7a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.494104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f20fd-c305-4122-b87b-46bcfeba7a73 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.494769 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f20fd-c305-4122-b87b-46bcfeba7a73 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:23.495063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f20fd-c305-4122-b87b-46bcfeba7a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:23.497379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f20fd-c305-4122-b87b-46bcfeba7a73 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:23.509194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 156/306] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:23.512604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1e2d-cd4b-4009-913d-05b11d291c7f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.512604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1e2d-cd4b-4009-913d-05b11d291c7f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.512604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1e2d-cd4b-4009-913d-05b11d291c7f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.513303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 157/307] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:23.520297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7638c8-8b9a-4bcd-8dc7-0375ee762638 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.520644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7638c8-8b9a-4bcd-8dc7-0375ee762638 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.520878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7638c8-8b9a-4bcd-8dc7-0375ee762638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.540003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-db7638c8-8b9a-4bcd-8dc7-0375ee762638 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:23.541323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-db7638c8-8b9a-4bcd-8dc7-0375ee762638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:23.541967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:23.546630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c7c8-561a-4a70-8bf9-ad71ee8bf484 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.595761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.596402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.596720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.596987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] PATH_INFO: `/v3/users/16b71ff0c6e9418bbb7beea419e890a2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.614311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=16b71ff0c6e9418bbb7beea419e890a2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:23.615957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:23.617860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 151/308] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:23 2022] GET /identity/v3/users/16b71ff0c6e9418bbb7beea419e890a2 => generated 293 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:23.617860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.636785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 158/309] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:23.637944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7638c8-8b9a-4bcd-8dc7-0375ee762638 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.777432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7652f5b-ea5f-4b18-a9d0-d658c070b3c9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.807682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:23.821972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.822799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.823175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.823568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] PATH_INFO: `/v3/users/34b32c79aff04a32a47c3d4d776ee179` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.825012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] RBAC: Authorizing `identity:get_user(user_id=34b32c79aff04a32a47c3d4d776ee179)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:23.826823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:23.828737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 152/310] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:23 2022] GET /identity/v3/users/34b32c79aff04a32a47c3d4d776ee179 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:23.834828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.846082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7638c8-8b9a-4bcd-8dc7-0375ee762638 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:23.869631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:23.929658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:23.931596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:23.932320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:23.932852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:23.934235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] RBAC: Adding query filter params (role.id=89cf360b7f5f43ec85bcf8dea35985dd, user.id=16b71ff0c6e9418bbb7beea419e890a2, scope.system=all) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:23.934781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:23.937566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:23.946309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 159/311] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 13 17:47:23 2022] GET /identity/v3/role_assignments?user.id=16b71ff0c6e9418bbb7beea419e890a2&scope.system=all&role.id=89cf360b7f5f43ec85bcf8dea35985dd => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:23.951308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:23.957011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7545815-3e7c-45bf-97f3-4f5d3d64311f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:24.050916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:24.054357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:24.055349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:24.056045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] PATH_INFO: `/v3/system/users/34b32c79aff04a32a47c3d4d776ee179/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:24.061525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=34b32c79aff04a32a47c3d4d776ee179, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:24.064539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:24.087135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 153/312] 10.222.0.43 () {58 vars in 1334 bytes} [Tue Sep 13 17:47:23 2022] PUT /identity/v3/system/users/34b32c79aff04a32a47c3d4d776ee179/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:24.117832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e0873-eb62-4411-9eb1-c4f79cbbb34a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:24.196137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:24.197590 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:24.198296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:24.198922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:24.220932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:24.230997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:24.233086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:24.235008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:24.238801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 160/313] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:24.246921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:24.363809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:24.379204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd56f18-0403-4cec-a121-68b91a751491 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:24.442590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:24.443651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:24.444203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:24.444737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:24.445974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:24.446517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:24.449225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:24.466712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 154/314] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:24.472918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:24.485935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:24.613732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088663f7-50fa-4abe-bfcb-aea6ede4289a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:24.684938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:24.684938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:24.684938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:24.684938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] PATH_INFO: `/v3/users/7720c048489d47349e2fed6f452ced0e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:24.685524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] RBAC: Authorizing `identity:get_user(user_id=7720c048489d47349e2fed6f452ced0e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:24.685524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:24.687036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 161/315] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:24 2022] GET /identity/v3/users/7720c048489d47349e2fed6f452ced0e => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:24.700018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:24.901381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163f2af-d0f9-4503-a615-e12449598310 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:24.966899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:24.967837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:24.968307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:24.968837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:25.010252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:25.100865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:25.103517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:25.278515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 155/316] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:25.287990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:25.391549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:25.441759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88170-ff83-4f77-af8a-76c4f010471f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:25.495955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f881fc-8429-46ea-85a9-cc67598d49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:25.496682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f881fc-8429-46ea-85a9-cc67598d49c3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:25.497131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f881fc-8429-46ea-85a9-cc67598d49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:25.497554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f881fc-8429-46ea-85a9-cc67598d49c3 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:25.498935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f881fc-8429-46ea-85a9-cc67598d49c3 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:25.500873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f881fc-8429-46ea-85a9-cc67598d49c3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:25.503445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 162/317] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:25 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:25.509811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:25.545242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec24500-861e-44da-83cf-95b28211db63 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:25.545652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec24500-861e-44da-83cf-95b28211db63 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:25.546146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec24500-861e-44da-83cf-95b28211db63 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:25.548871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 163/318] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:25.558032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadedabe-c54a-4a6a-aba1-40ad2365e7a9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:25.558465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadedabe-c54a-4a6a-aba1-40ad2365e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:25.558889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadedabe-c54a-4a6a-aba1-40ad2365e7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:25.574561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:25.585419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cadedabe-c54a-4a6a-aba1-40ad2365e7a9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:25.587451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cadedabe-c54a-4a6a-aba1-40ad2365e7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:25.631990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfd1a9-c142-41ed-854c-7f50bfc5093b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:25.671812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:25.672846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:25.673315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:25.673768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:25.674856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] RBAC: Adding query filter params (role.id=89cf360b7f5f43ec85bcf8dea35985dd, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81, user.id=7720c048489d47349e2fed6f452ced0e) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:25.675369 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:25.677107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:25.690228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 156/319] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:47:25 2022] GET /identity/v3/role_assignments?user.id=7720c048489d47349e2fed6f452ced0e&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=89cf360b7f5f43ec85bcf8dea35985dd => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:25.704648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 164/320] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:25.965110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:26.092059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ae4a9-c3fb-47d1-b54f-df7cb3eec990 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:26.121167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:26.150070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00e9e1-17ec-4ecb-8e55-8ef8e9f8478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:26.150798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e9e1-17ec-4ecb-8e55-8ef8e9f8478d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:26.151203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e9e1-17ec-4ecb-8e55-8ef8e9f8478d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:26.151496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e9e1-17ec-4ecb-8e55-8ef8e9f8478d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:26.152934 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00e9e1-17ec-4ecb-8e55-8ef8e9f8478d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:26.154544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00e9e1-17ec-4ecb-8e55-8ef8e9f8478d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:26.156731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 157/321] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:26.165241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadedabe-c54a-4a6a-aba1-40ad2365e7a9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:26.307012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:26.359240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadedabe-c54a-4a6a-aba1-40ad2365e7a9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:26.416999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edfb6b-535a-498e-b1d3-de31b5755df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:26.418031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edfb6b-535a-498e-b1d3-de31b5755df3 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:26.418455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edfb6b-535a-498e-b1d3-de31b5755df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:26.418935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edfb6b-535a-498e-b1d3-de31b5755df3 None admin] PATH_INFO: `/v3/groups` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:26.420079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edfb6b-535a-498e-b1d3-de31b5755df3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:26.422890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edfb6b-535a-498e-b1d3-de31b5755df3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:26.427103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f847-6306-49f6-bd72-e01f0e70d82e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:26.427397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f847-6306-49f6-bd72-e01f0e70d82e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:26.427666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f847-6306-49f6-bd72-e01f0e70d82e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:26.429388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 158/322] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:26.435081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ea5c1-b5c4-4d0e-bfc9-a148e7fde553 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:26.435323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ea5c1-b5c4-4d0e-bfc9-a148e7fde553 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:26.435685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ea5c1-b5c4-4d0e-bfc9-a148e7fde553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:26.446506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 165/323] 10.222.0.43 () {60 vars in 1134 bytes} [Tue Sep 13 17:47:26 2022] POST /identity/v3/groups => generated 226 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:26.474822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2a6ea5c1-b5c4-4d0e-bfc9-a148e7fde553 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:26.476249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2a6ea5c1-b5c4-4d0e-bfc9-a148e7fde553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:26.596608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 159/324] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:26.754944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c70869-6358-4ef2-861f-63e6233e5b4d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:26.755246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c70869-6358-4ef2-861f-63e6233e5b4d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:26.755613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c70869-6358-4ef2-861f-63e6233e5b4d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:26.757073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 166/325] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:26.763768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df99ea-d916-4cd1-b20b-3d2804f24105 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:26.764059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df99ea-d916-4cd1-b20b-3d2804f24105 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:26.764367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df99ea-d916-4cd1-b20b-3d2804f24105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:26.789781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-69df99ea-d916-4cd1-b20b-3d2804f24105 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:26.792393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-69df99ea-d916-4cd1-b20b-3d2804f24105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:26.906795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c70869-6358-4ef2-861f-63e6233e5b4d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:26.928359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 160/326] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:27.120284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c70869-6358-4ef2-861f-63e6233e5b4d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:27.196049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:27.197612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:27.198395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:27.199227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:27.204131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df99ea-d916-4cd1-b20b-3d2804f24105 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:27.208373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:27.232821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:27.242707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:27.243688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:27.244944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:27.245567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 167/327] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:27.253595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:27.299355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:27.426115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df99ea-d916-4cd1-b20b-3d2804f24105 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:27.472607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff2aa-275a-4c4d-975e-61355ca8ec14 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:27.499281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:27.500397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:27.500886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:27.501354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:27.521648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:27.531283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:27.533190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:27.535045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 161/328] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:27.548139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:27.573504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:27.574930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:27.575702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:27.576384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:27.578061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:27.578863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:27.582130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:27.602385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 168/329] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:27.612615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:27.731460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:27.777178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cdbfa-bda3-4734-833e-025c6e7c5727 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:27.782003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:27.846223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:27.847702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:27.848458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:27.849790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:27.851029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:27.852619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:27.853916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c29fe9-2f1b-4915-9c3d-5bfbdcff77f3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:27.857173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:27.876805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 162/330] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:27.888896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:27.910499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:27.911644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:27.912130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:27.912653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] PATH_INFO: `/v3/users/16b71ff0c6e9418bbb7beea419e890a2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:27.914200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] RBAC: Authorizing `identity:get_user(user_id=16b71ff0c6e9418bbb7beea419e890a2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:27.917357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:27.919598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 169/331] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:27 2022] GET /identity/v3/users/16b71ff0c6e9418bbb7beea419e890a2 => generated 293 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:27.930071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:28.113747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e1150-5917-44d0-8e42-4e919e31c7c6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:28.158353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3579e-a421-4c60-962a-6b5d427a308f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:28.166182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:28.167192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:28.167705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:28.168175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] PATH_INFO: `/v3/users/34b32c79aff04a32a47c3d4d776ee179` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:28.169608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] RBAC: Authorizing `identity:get_user(user_id=34b32c79aff04a32a47c3d4d776ee179)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:28.171373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:28.173301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 163/332] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:27 2022] GET /identity/v3/users/34b32c79aff04a32a47c3d4d776ee179 => generated 293 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:28.185030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:28.217038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:28.237271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c8cd2-f166-4fdd-9805-f395cfb02992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:28.239880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c8cd2-f166-4fdd-9805-f395cfb02992 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:28.240178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c8cd2-f166-4fdd-9805-f395cfb02992 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:28.240474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c8cd2-f166-4fdd-9805-f395cfb02992 None admin] PATH_INFO: `/v3/system/users/16b71ff0c6e9418bbb7beea419e890a2/roles/89cf360b7f5f43ec85bcf8dea35985dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:28.241019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:28.245950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c8cd2-f166-4fdd-9805-f395cfb02992 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=16b71ff0c6e9418bbb7beea419e890a2, role_id=89cf360b7f5f43ec85bcf8dea35985dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:28.246268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c8cd2-f166-4fdd-9805-f395cfb02992 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:28.272099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 170/333] 10.222.0.43 () {58 vars in 1334 bytes} [Tue Sep 13 17:47:27 2022] PUT /identity/v3/system/users/16b71ff0c6e9418bbb7beea419e890a2/roles/89cf360b7f5f43ec85bcf8dea35985dd => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:28.338392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739da355-c637-4e1c-ba07-41ada992dbb5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:28.382708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2d88-bd0c-4355-b76e-cdd631b6c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:28.383370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2d88-bd0c-4355-b76e-cdd631b6c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:28.383541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2d88-bd0c-4355-b76e-cdd631b6c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:28.383816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2d88-bd0c-4355-b76e-cdd631b6c6c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:28.384583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2d88-bd0c-4355-b76e-cdd631b6c6c2 None admin] RBAC: Adding query filter params (role.id=5c592b5b170d48b0825640b8283425de, user.id=34b32c79aff04a32a47c3d4d776ee179, scope.system=all) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:28.384888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2d88-bd0c-4355-b76e-cdd631b6c6c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:28.386377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2d88-bd0c-4355-b76e-cdd631b6c6c2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:28.401663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 164/334] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 13 17:47:28 2022] GET /identity/v3/role_assignments?user.id=34b32c79aff04a32a47c3d4d776ee179&scope.system=all&role.id=5c592b5b170d48b0825640b8283425de => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:29.758105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06aca2-5b4e-4add-8b8a-d2350526a0c3 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:29.759090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06aca2-5b4e-4add-8b8a-d2350526a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:29.759580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06aca2-5b4e-4add-8b8a-d2350526a0c3 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:29.761458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 171/335] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:29.768582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effee02f-cb92-47c4-ad7a-b4e08c419442 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:29.769520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effee02f-cb92-47c4-ad7a-b4e08c419442 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:29.769991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effee02f-cb92-47c4-ad7a-b4e08c419442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:29.789847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-effee02f-cb92-47c4-ad7a-b4e08c419442 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:29.791420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-effee02f-cb92-47c4-ad7a-b4e08c419442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:29.896661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 165/336] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:30.427157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06aca2-5b4e-4add-8b8a-d2350526a0c3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:30.567295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20176a7-80a8-4772-8185-14ad32af2d50 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:30.568075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20176a7-80a8-4772-8185-14ad32af2d50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:30.568851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20176a7-80a8-4772-8185-14ad32af2d50 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:30.571558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 166/337] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:30.579829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5afc27-fcaa-45ba-8a2a-15cef6bb8cd3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:30.580791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5afc27-fcaa-45ba-8a2a-15cef6bb8cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:30.581294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5afc27-fcaa-45ba-8a2a-15cef6bb8cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:30.588285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06aca2-5b4e-4add-8b8a-d2350526a0c3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:30.606180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3b5afc27-fcaa-45ba-8a2a-15cef6bb8cd3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:30.608700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3b5afc27-fcaa-45ba-8a2a-15cef6bb8cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:30.654386 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:30.663249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400c057-3cd4-4155-b8d6-05e507a3e640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:30.664979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400c057-3cd4-4155-b8d6-05e507a3e640 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:30.666924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400c057-3cd4-4155-b8d6-05e507a3e640 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:30.668256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400c057-3cd4-4155-b8d6-05e507a3e640 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:30.702247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400c057-3cd4-4155-b8d6-05e507a3e640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5400c057-3cd4-4155-b8d6-05e507a3e640 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:30.712859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:30.714519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:30.716010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:30.720384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 172/338] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:30.721975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d7575-2dac-4856-8c3a-ad6af1391c9e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:30.722268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d7575-2dac-4856-8c3a-ad6af1391c9e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:30.722515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d7575-2dac-4856-8c3a-ad6af1391c9e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:30.724332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 173/339] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:30.725293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d7575-2dac-4856-8c3a-ad6af1391c9e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:30.783719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 167/340] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:30.785584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fe3d4-f2b3-4f5b-940d-9cc86a3ae957 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:30.785790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fe3d4-f2b3-4f5b-940d-9cc86a3ae957 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:30.785920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fe3d4-f2b3-4f5b-940d-9cc86a3ae957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:30.836221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3b0fe3d4-f2b3-4f5b-940d-9cc86a3ae957 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:30.847281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3b0fe3d4-f2b3-4f5b-940d-9cc86a3ae957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:30.852560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:31.003285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 168/341] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:31.063066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d7575-2dac-4856-8c3a-ad6af1391c9e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:31.132466 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:31.133511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:31.133984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:31.134460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:31.135679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:31.136183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:31.139076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:31.157510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 174/342] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:31.163829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fe3d4-f2b3-4f5b-940d-9cc86a3ae957 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:31.191516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:31.335059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:31.384400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fe3d4-f2b3-4f5b-940d-9cc86a3ae957 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:31.436399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c624878-e329-436a-bbf8-83ee680ce239 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:31.459456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:31.461494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:31.462239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:31.462511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:31.462775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:31.514827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376f215-c894-4b16-91e2-6ca55da729e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:31.515479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376f215-c894-4b16-91e2-6ca55da729e5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:31.515754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376f215-c894-4b16-91e2-6ca55da729e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:31.516019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376f215-c894-4b16-91e2-6ca55da729e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:31.539706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376f215-c894-4b16-91e2-6ca55da729e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f376f215-c894-4b16-91e2-6ca55da729e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:31.552198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:31.553877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 175/343] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:31.555279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bdbfc-a959-40b8-b3ca-b4602c05fac5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:31.557492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bdbfc-a959-40b8-b3ca-b4602c05fac5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:31.557492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bdbfc-a959-40b8-b3ca-b4602c05fac5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:31.563584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 176/344] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:31.567206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bdbfc-a959-40b8-b3ca-b4602c05fac5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:31.600381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:31.601878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:31.734265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bdbfc-a959-40b8-b3ca-b4602c05fac5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:31.781056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:31.786158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c12172-76c2-43b1-8690-9f7b108d3784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:31.786841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c12172-76c2-43b1-8690-9f7b108d3784 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:31.787316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c12172-76c2-43b1-8690-9f7b108d3784 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:31.788398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c12172-76c2-43b1-8690-9f7b108d3784 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:31.790368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c12172-76c2-43b1-8690-9f7b108d3784 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:31.790567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c12172-76c2-43b1-8690-9f7b108d3784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:31.792041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c12172-76c2-43b1-8690-9f7b108d3784 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:31.806735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 177/345] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:31.808377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64d996-3c70-4bae-879e-bc26d9b3af77 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:31.808761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64d996-3c70-4bae-879e-bc26d9b3af77 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:31.809001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64d996-3c70-4bae-879e-bc26d9b3af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:31.826651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 169/346] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 17:47:31.829208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:31.831653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1f64d996-3c70-4bae-879e-bc26d9b3af77 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:31.839030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1f64d996-3c70-4bae-879e-bc26d9b3af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:31.899650 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:31.964121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ea6c2-bc9e-4fbe-b870-ee358b3e8cef None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:31.993963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 178/347] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:31.995817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64d996-3c70-4bae-879e-bc26d9b3af77 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:32.014091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a991b-bebb-430f-8039-e16375407670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:32.014757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a991b-bebb-430f-8039-e16375407670 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:32.015032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a991b-bebb-430f-8039-e16375407670 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:32.015297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a991b-bebb-430f-8039-e16375407670 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:32.034871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a991b-bebb-430f-8039-e16375407670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-106a991b-bebb-430f-8039-e16375407670 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:32.040468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:32.042240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:32.043843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:32.047520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 170/348] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:32.048496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a991b-bebb-430f-8039-e16375407670 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:32.204909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:32.333669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64d996-3c70-4bae-879e-bc26d9b3af77 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:32.340003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a991b-bebb-430f-8039-e16375407670 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:32.388898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:32.389551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:32.389826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:32.390094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:32.391318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:32.392883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:32.394726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 171/349] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:32 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:32.395606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:32.426096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:32.428249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:32.429281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:32.430204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:32.498615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:32.590973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:32.591451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d28b7-5999-491e-ac5d-828475be6396 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:32.593715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:32.611737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:32.659593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:32.661029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:32.661546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:32.662055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:32.663403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:32.663961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:32.666453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:32.681605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 172/350] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:32.682779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:32.789903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 179/351] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:32.793957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:32.871761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a49154-54d2-441e-ae54-676ee1026552 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:32.878604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:32.929333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f23d0-ee97-464f-9958-ab06e0e82f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:32.930128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438447e4-f834-47ff-a066-f2c472386283 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:32.930569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f23d0-ee97-464f-9958-ab06e0e82f16 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:32.930761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f23d0-ee97-464f-9958-ab06e0e82f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:32.931204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f23d0-ee97-464f-9958-ab06e0e82f16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:32.947396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f23d0-ee97-464f-9958-ab06e0e82f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-240f23d0-ee97-464f-9958-ab06e0e82f16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:32.952996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:32.954273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:32.955602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 173/352] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:32.960565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fedc0-9cab-46c8-bc7d-3a6fe7b7d894 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:32.960941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fedc0-9cab-46c8-bc7d-3a6fe7b7d894 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:32.961160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fedc0-9cab-46c8-bc7d-3a6fe7b7d894 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:32.963468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 174/353] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:32.965459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fedc0-9cab-46c8-bc7d-3a6fe7b7d894 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:32.981971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:32.982925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:32.983207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:32.983481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:32.990326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:32.991897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:32.994741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 180/354] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:32 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:32.995669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:33.013486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:33.119438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fedc0-9cab-46c8-bc7d-3a6fe7b7d894 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:33.129951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a407f-45c5-4764-8704-fdd6dff4fad2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:33.163701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:33.170676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760be91-9e02-4f53-9ab7-0da93b33876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:33.171890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760be91-9e02-4f53-9ab7-0da93b33876b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.172169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760be91-9e02-4f53-9ab7-0da93b33876b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.172479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760be91-9e02-4f53-9ab7-0da93b33876b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.181935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018fabb-3abc-4242-badc-18d79d4d37ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:33.183214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fabb-3abc-4242-badc-18d79d4d37ea None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.183491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fabb-3abc-4242-badc-18d79d4d37ea None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.183758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fabb-3abc-4242-badc-18d79d4d37ea None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.185150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fabb-3abc-4242-badc-18d79d4d37ea None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:33.186674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fabb-3abc-4242-badc-18d79d4d37ea None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:33.188382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 181/355] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:32 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:33.189610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49544ff1-205a-4e2c-96fd-fd71a933fd75 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.189884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49544ff1-205a-4e2c-96fd-fd71a933fd75 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.190154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49544ff1-205a-4e2c-96fd-fd71a933fd75 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.191375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 182/356] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:33.192077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49544ff1-205a-4e2c-96fd-fd71a933fd75 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:33.265974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760be91-9e02-4f53-9ab7-0da93b33876b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:33.267456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760be91-9e02-4f53-9ab7-0da93b33876b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:33.306890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:33.358397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49544ff1-205a-4e2c-96fd-fd71a933fd75 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:33.398343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 175/357] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:33.399518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06917-7564-4f4c-acda-5cc0ec8d8d7f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.399839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06917-7564-4f4c-acda-5cc0ec8d8d7f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.400093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e06917-7564-4f4c-acda-5cc0ec8d8d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.411205 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:33.411835 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.412102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.412372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.413159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:33.413458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:33.414913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:33.422919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-14e06917-7564-4f4c-acda-5cc0ec8d8d7f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:33.424127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14e06917-7564-4f4c-acda-5cc0ec8d8d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:33.428659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 183/358] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:33.428659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:33.513545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:33.529710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 176/359] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:33.530598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06917-7564-4f4c-acda-5cc0ec8d8d7f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:33.593496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dafc6-902d-49c4-98e5-30d79633a836 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:33.621517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:33.647009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afc48f-7dcb-4b90-b9f4-6ada6b84fe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:33.647693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afc48f-7dcb-4b90-b9f4-6ada6b84fe24 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.647957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afc48f-7dcb-4b90-b9f4-6ada6b84fe24 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.649181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afc48f-7dcb-4b90-b9f4-6ada6b84fe24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.649181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afc48f-7dcb-4b90-b9f4-6ada6b84fe24 None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=a5a5bade20834645af2806f86402c383) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:33.649503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afc48f-7dcb-4b90-b9f4-6ada6b84fe24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:33.650807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afc48f-7dcb-4b90-b9f4-6ada6b84fe24 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:33.657339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 184/360] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:33 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:33.658436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9c030-42ac-473f-aa28-5365d0949fcd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.658716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9c030-42ac-473f-aa28-5365d0949fcd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.662397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9c030-42ac-473f-aa28-5365d0949fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.685367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-27e9c030-42ac-473f-aa28-5365d0949fcd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:33.686601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-27e9c030-42ac-473f-aa28-5365d0949fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:33.702114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e06917-7564-4f4c-acda-5cc0ec8d8d7f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:33.756674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:33.757336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.757610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.757877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.759417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:33.761046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:33.763524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 177/361] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:33 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:33.764368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:33.792924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 185/362] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:33.794041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9c030-42ac-473f-aa28-5365d0949fcd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:33.890970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23a16b-47e4-48fa-b73e-242b7bd6753a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:33.894114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:33.927843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:33.937563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:33.938500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:33.938913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:33.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:33.940909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:33.942711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:33.945233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 178/363] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:33 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:33.946345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:34.001617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9c030-42ac-473f-aa28-5365d0949fcd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:34.059535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58300658-0b7e-4479-8026-038945d47678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:34.061013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58300658-0b7e-4479-8026-038945d47678 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:34.061457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58300658-0b7e-4479-8026-038945d47678 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:34.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58300658-0b7e-4479-8026-038945d47678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:34.172942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c1d61-f6f7-4a94-9f14-547e1eadb230 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:34.206102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58300658-0b7e-4479-8026-038945d47678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:34.209257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58300658-0b7e-4479-8026-038945d47678 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:34.236939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:34.237721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:34.240006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:34.240006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:34.240006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=a5a5bade20834645af2806f86402c383) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:34.240463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:34.243132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:34.253540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 179/364] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:33 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=5c592b5b170d48b0825640b8283425de => generated 260 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:34.254927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:34.288584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:34.351947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:34.380361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 186/365] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:34.381857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58300658-0b7e-4479-8026-038945d47678 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:34.406745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be28604-5b6d-47cb-be57-bba9b309e790 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:34.448821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:34.449504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:34.449783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:34.450056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:34.451277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:34.452803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:34.454881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 180/366] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:34 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:34.455716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:34.537732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58300658-0b7e-4479-8026-038945d47678 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:34.589976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:34.591736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:34.592532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:34.593302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:34.595813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177b96a-e52e-47d1-836b-9e41011e7f88 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:34.614594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:34.624947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:34.626657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 17:47:34.628190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:34.630139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 187/367] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:34.630842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:34.649156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:34.654274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:34.661115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:34.662153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:34.662153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:34.662573 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:34.679894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:34.687620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:34.689635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 181/368] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:34.691427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:34.794139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b267f44-563a-4e39-9d48-d18f347428d1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:34.842840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84124450-a867-4b44-a3f5-9a888d410a47 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:34.844699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:34.845470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:34.845750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:34.846025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:34.847262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:34.848891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:34.851045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 188/369] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:34 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:34.851940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:34.895629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:34.896318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:34.896567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:34.896885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:34.897633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:34.897938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:34.899432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:34.905870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 182/370] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:34 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=5c592b5b170d48b0825640b8283425de => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:34.913665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:35.002578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:35.042533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:35.060829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4ee7f-ab41-415a-b1e5-68dcb61fe131 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:35.145011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29eedb-96d1-46ee-93f6-1c0a29a5ef43 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:35.148622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:35.149332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:35.149609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:35.149870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:35.150611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:35.150911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:35.152401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:35.174068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 189/371] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:35.175493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:35.205904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:35.206528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:35.206816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:35.207061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:35.207740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:35.208038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:35.209557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:35.222205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 183/372] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:35.222916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:35.363655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a268888-38c9-4711-87d0-3144684a44a0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:35.394707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:35.407885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:35.408610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:35.408892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:35.409164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] PATH_INFO: `/v3/users/16b71ff0c6e9418bbb7beea419e890a2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:35.410383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] RBAC: Authorizing `identity:get_user(user_id=16b71ff0c6e9418bbb7beea419e890a2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:35.411867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:35.418768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 184/373] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:47:35 2022] GET /identity/v3/users/16b71ff0c6e9418bbb7beea419e890a2 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:35.419792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:35.475324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089b424-d54f-4312-9824-28b31e98d699 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:35.545601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7534c-27f0-4f43-976d-fc11c1f95036 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:35.564076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:35.566313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:35.567084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:35.567823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:35.571425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:35.576349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:35.582481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 190/374] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:35 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:35.588494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:35.588494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:35.588494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:35.588494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:35.589123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:35.613100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:35.693794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:35.727831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:35.730556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:35.804602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593ffc8-8fce-4f76-9409-0a8886ae6cbd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:35.879355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:35.879981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:35.880330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:35.880656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:35.881495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] RBAC: Adding query filter params (role.id=89cf360b7f5f43ec85bcf8dea35985dd, user.id=16b71ff0c6e9418bbb7beea419e890a2, scope.system=all) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:35.881798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:35.883295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:35.915971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 191/375] 10.222.0.43 () {56 vars in 1308 bytes} [Tue Sep 13 17:47:35 2022] GET /identity/v3/role_assignments?user.id=16b71ff0c6e9418bbb7beea419e890a2&scope.system=all&role.id=89cf360b7f5f43ec85bcf8dea35985dd => generated 510 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:35.917041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:35.948592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 185/376] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:35.958343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:36.073348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:36.097283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:36.114621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bda15-dfce-43ab-a042-cb462aa6c218 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:36.119893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32868c16-dd0a-4321-ab3b-951808df8ed4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:36.153020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:36.153648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:36.153945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:36.154178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:36.154957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:36.155265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:36.156806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:36.163364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 192/377] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:35 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:36.165723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:36.166404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:36.166685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:36.166960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] PATH_INFO: `/v3/projects/c326683f56fe42d09b126fc368517cb3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:36.168308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] RBAC: Authorizing `identity:get_project(project_id=c326683f56fe42d09b126fc368517cb3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:36.169880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:36.176464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 186/378] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:35 2022] GET /identity/v3/projects/c326683f56fe42d09b126fc368517cb3 => generated 344 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:36.178352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:36.329368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8f334-cee5-48a4-a113-94fde1ee25e5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:36.384361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:36.393505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b827e6-efa9-4bf3-b729-dab279085ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:36.394538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b827e6-efa9-4bf3-b729-dab279085ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:36.395682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b827e6-efa9-4bf3-b729-dab279085ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:36.396800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b827e6-efa9-4bf3-b729-dab279085ef3 None admin] PATH_INFO: `/v3/groups/4787ca47c646401a8181ee3e0e19e19b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:36.404128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b827e6-efa9-4bf3-b729-dab279085ef3 None admin] RBAC: Authorizing `identity:get_group(group_id=4787ca47c646401a8181ee3e0e19e19b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:36.406740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b827e6-efa9-4bf3-b729-dab279085ef3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:36.410036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 193/379] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:36 2022] GET /identity/v3/groups/4787ca47c646401a8181ee3e0e19e19b => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:36.417036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:36.547635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb05c49-7461-4d72-a9f7-4c1ef6eb16fe None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:36.588016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f0e4d-8765-450d-9273-6faa0da6671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:36.588600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f0e4d-8765-450d-9273-6faa0da6671a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:36.588880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f0e4d-8765-450d-9273-6faa0da6671a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:36.589152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f0e4d-8765-450d-9273-6faa0da6671a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:36.589934 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f0e4d-8765-450d-9273-6faa0da6671a None admin] RBAC: Adding query filter params (group.id=4787ca47c646401a8181ee3e0e19e19b, role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=c326683f56fe42d09b126fc368517cb3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:36.590250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f0e4d-8765-450d-9273-6faa0da6671a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:36.591798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f0e4d-8765-450d-9273-6faa0da6671a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:36.599033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 187/380] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:36 2022] GET /identity/v3/role_assignments?group.id=4787ca47c646401a8181ee3e0e19e19b&scope.project.id=c326683f56fe42d09b126fc368517cb3&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:37.203000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbe8bf-ce51-4dc1-b015-65d72d325cb3 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:37.203575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbe8bf-ce51-4dc1-b015-65d72d325cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:37.203575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbe8bf-ce51-4dc1-b015-65d72d325cb3 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:37.205304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 194/381] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:37.210891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cc282-c6ad-431b-ba4a-18b0f9d180ee None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:37.211399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cc282-c6ad-431b-ba4a-18b0f9d180ee None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:37.211503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cc282-c6ad-431b-ba4a-18b0f9d180ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:37.232329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8e1cc282-c6ad-431b-ba4a-18b0f9d180ee None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:37.233670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8e1cc282-c6ad-431b-ba4a-18b0f9d180ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:37.247472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:37.331278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 188/382] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:37.727138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbe8bf-ce51-4dc1-b015-65d72d325cb3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:37.758214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4265c65-1094-47f6-afc7-c04932d2c30e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:37.758549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4265c65-1094-47f6-afc7-c04932d2c30e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:37.758762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4265c65-1094-47f6-afc7-c04932d2c30e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:37.760537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 189/383] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:37.766714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298dc7-7a58-4f88-a6af-c5523574ccfe None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:37.767026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298dc7-7a58-4f88-a6af-c5523574ccfe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:37.767260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298dc7-7a58-4f88-a6af-c5523574ccfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:37.798684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3b298dc7-7a58-4f88-a6af-c5523574ccfe None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:37.799935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3b298dc7-7a58-4f88-a6af-c5523574ccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:37.935008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 190/384] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 17:47:37.973436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbe8bf-ce51-4dc1-b015-65d72d325cb3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:38.014129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:38.014698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:38.014977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:38.015242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:38.027640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:38.033216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:38.034167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:38.034937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 195/385] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:38.040772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298dc7-7a58-4f88-a6af-c5523574ccfe None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:38.153930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:38.187185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298dc7-7a58-4f88-a6af-c5523574ccfe None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:38.239584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:38.240254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:38.240635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:38.240826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:38.241521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:38.241827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:38.243265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:38.252935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 191/386] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:38.259214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:38.341731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:38.402941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd0243-b409-4c04-8f25-ed64293734f8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:38.449069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ce274-aa4c-4d7e-94a1-76b36b0a4773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:38.449779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ce274-aa4c-4d7e-94a1-76b36b0a4773 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:38.450107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ce274-aa4c-4d7e-94a1-76b36b0a4773 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:38.450467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ce274-aa4c-4d7e-94a1-76b36b0a4773 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:38.454302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ce274-aa4c-4d7e-94a1-76b36b0a4773 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:38.455992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ce274-aa4c-4d7e-94a1-76b36b0a4773 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:38.458134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 196/387] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:38 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:38.464019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:38.596777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944bfe0-5330-4e04-bb0e-a8027fa02be4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:38.596953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944bfe0-5330-4e04-bb0e-a8027fa02be4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:38.597239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944bfe0-5330-4e04-bb0e-a8027fa02be4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:38.598830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 197/388] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:38.606246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f61ac-0dff-419d-b581-68486d9cbd92 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:38.606544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f61ac-0dff-419d-b581-68486d9cbd92 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:38.606798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f61ac-0dff-419d-b581-68486d9cbd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:38.613461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf91a-e903-441e-a38c-b0d6d22d9ed4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:38.632556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a72f61ac-0dff-419d-b581-68486d9cbd92 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:38.633980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a72f61ac-0dff-419d-b581-68486d9cbd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:38.698719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:38.699841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:38.700843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:38.703155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:38.705165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:38.707314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:38.710757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 192/389] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:38 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:38.712046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:38.733734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:38.772589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 198/390] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:38.773825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f61ac-0dff-419d-b581-68486d9cbd92 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:38.850453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd361b0-c559-45d6-bb89-1898554464ad None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:38.857678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:38.899693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:38.900356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:38.900666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:38.900943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:38.913901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:38.919779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:38.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:38.921638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 193/391] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:38.927603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:39.010846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:39.025860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f61ac-0dff-419d-b581-68486d9cbd92 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:39.088300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebddd2-fd55-466e-a85f-5336623e76a6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:39.122510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:39.123905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:39.125019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:39.125513 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383/groups/ca513b2088c2432395c23921087b2075/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:39.129019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bade20834645af2806f86402c383, group_id=ca513b2088c2432395c23921087b2075, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:39.132472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:39.160942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:39.161414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 199/392] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 13 17:47:38 2022] PUT /identity/v3/projects/a5a5bade20834645af2806f86402c383/groups/ca513b2088c2432395c23921087b2075/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:39.162347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:39.162869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:39.163145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:39.163920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:39.165353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:39.165966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:39.169783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:39.192462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 194/393] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:39.204888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:39.445378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b336dff-da1e-4461-bf94-afefe83dfe76 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:39.470240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a22603-72af-4abb-98c0-bddd12e93287 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:39.498893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:39.510180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:39.526346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833b568-bb62-4889-b4e1-6e68787e7829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:39.528020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833b568-bb62-4889-b4e1-6e68787e7829 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:39.528876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833b568-bb62-4889-b4e1-6e68787e7829 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:39.529657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833b568-bb62-4889-b4e1-6e68787e7829 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:39.532049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833b568-bb62-4889-b4e1-6e68787e7829 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:39.535164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833b568-bb62-4889-b4e1-6e68787e7829 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:39.539337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 195/394] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:39 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:39.541957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131a60d-3f99-42d5-87e2-eb8173378e08 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:39.542690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131a60d-3f99-42d5-87e2-eb8173378e08 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:39.543399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131a60d-3f99-42d5-87e2-eb8173378e08 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:39.546376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 196/395] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:39.546840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc22617-7b54-44ac-9ad6-ca26c7c4ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:39.547533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc22617-7b54-44ac-9ad6-ca26c7c4ad22 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:39.547808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc22617-7b54-44ac-9ad6-ca26c7c4ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:39.548075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc22617-7b54-44ac-9ad6-ca26c7c4ad22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:39.553439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131a60d-3f99-42d5-87e2-eb8173378e08 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:39.563292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc22617-7b54-44ac-9ad6-ca26c7c4ad22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-fdc22617-7b54-44ac-9ad6-ca26c7c4ad22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:39.568271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:39.570084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:39.571603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:39.575715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 200/396] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:39.580960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415ea0c-d7c2-47ff-9402-d3265b119858 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:39.580960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415ea0c-d7c2-47ff-9402-d3265b119858 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:39.580960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415ea0c-d7c2-47ff-9402-d3265b119858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:39.599598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4415ea0c-d7c2-47ff-9402-d3265b119858 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:39.601108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4415ea0c-d7c2-47ff-9402-d3265b119858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:39.734464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 201/397] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:39.736020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415ea0c-d7c2-47ff-9402-d3265b119858 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:39.757046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131a60d-3f99-42d5-87e2-eb8173378e08 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:39.848108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:39.848810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:39.849090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:39.849347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:39.854286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:39.857787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:39.864264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 197/398] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:39 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:39.873602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:39.987807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:40.001186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415ea0c-d7c2-47ff-9402-d3265b119858 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:40.004307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:40.043580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124c922-5f2e-44f3-8ac2-ca747c6b8d95 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:40.089588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57006efa-8b3b-417c-8939-f06a1711a4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:40.090278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57006efa-8b3b-417c-8939-f06a1711a4b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.090772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57006efa-8b3b-417c-8939-f06a1711a4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.091052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57006efa-8b3b-417c-8939-f06a1711a4b6 None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383/groups/ca513b2088c2432395c23921087b2075/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.093511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57006efa-8b3b-417c-8939-f06a1711a4b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5a5bade20834645af2806f86402c383, group_id=ca513b2088c2432395c23921087b2075, role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:40.095208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57006efa-8b3b-417c-8939-f06a1711a4b6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:40.124750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 198/399] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 13 17:47:39 2022] PUT /identity/v3/projects/a5a5bade20834645af2806f86402c383/groups/ca513b2088c2432395c23921087b2075/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 17:47:40.135541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:40.137445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.138213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.139056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.140681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:40.141544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:40.144610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:40.156991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a7db-ba10-4c81-9941-1e33cd9dba04 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.158007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a7db-ba10-4c81-9941-1e33cd9dba04 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.158773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397a7db-ba10-4c81-9941-1e33cd9dba04 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.161760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 199/400] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:40.171069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1745cb-9df6-4e99-8ca1-5ae5e2dc1ffb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.171488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1745cb-9df6-4e99-8ca1-5ae5e2dc1ffb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.171957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1745cb-9df6-4e99-8ca1-5ae5e2dc1ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.184534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 202/401] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:40.191420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:40.205612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2f1745cb-9df6-4e99-8ca1-5ae5e2dc1ffb None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:40.207554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2f1745cb-9df6-4e99-8ca1-5ae5e2dc1ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:40.352909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 200/402] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:40.354826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1745cb-9df6-4e99-8ca1-5ae5e2dc1ffb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:40.360258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e552-6738-4dc2-8fea-a68b2daf0615 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:40.404128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:40.405243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.405522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.405799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.407169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:40.408769 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:40.416584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 203/403] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:40 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:40.418719 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:40.460837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:40.513795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:40.579112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1745cb-9df6-4e99-8ca1-5ae5e2dc1ffb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:40.616726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:40.617703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.618197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.618663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.631399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:40.636767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:40.638049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:40.638908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:40.641428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 201/404] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:40.650512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:40.679238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90a862-9b1b-40ae-a658-713dab0c6e39 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:40.788574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:40.789552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.789955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.790356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.800761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:40.803726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:40.811460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 204/405] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:40 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:40.830215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:40.866378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a7e2d-6152-4997-8a80-78ca9e795d31 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:40.897211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:40.942221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:40.943341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:40.943895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:40.944378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:40.945294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:40.945887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:40.947450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:40.957831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 202/406] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:40.958881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:41.049111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:41.052998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4227b38-280b-4769-aa0a-1d6219d6695a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:41.096512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:41.097530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:41.097782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:41.098157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/groups/ca513b2088c2432395c23921087b2075/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:41.100291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] RBAC: Authorizing `identity:create_grant(project_id=16f3442e76ed418d936b9a5eb6f19e81, group_id=ca513b2088c2432395c23921087b2075, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:41.102057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:41.124159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 205/407] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 13 17:47:40 2022] PUT /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/groups/ca513b2088c2432395c23921087b2075/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:41.125620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:41.133726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062ccb0-0dbb-41fe-b41e-2750fc721e4e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:41.211582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:41.213293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:41.215315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:41.216225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:41.251334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:41.276095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:41.277756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:41.279199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:41.283264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 203/408] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:47:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:41.300258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:41.396006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94240-7d72-4aac-91a6-e7d23b1c3992 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:41.403918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:41.482123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:41.483462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:41.483778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:41.484031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:41.486392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:41.487915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:41.490014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 206/409] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:41 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:41.496178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:41.521623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782d57e-b03d-48a8-a7f0-37af4d7b691f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:41.568713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:41.569368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:41.569653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:41.569949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:41.570811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:41.571065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:41.572561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:41.573371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:41.587107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 204/410] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:41.593786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:41.781372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a30343-9156-4d6c-b73e-1d1f98a399c7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:41.852050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:41.853181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:41.853659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:41.854122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:41.857919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:41.859427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:41.862252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 207/411] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:41 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:41.868705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:41.932969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:41.936924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f78cf-9894-4fa8-a474-7171bffe4b7b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:42.004451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:42.004999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:42.005323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:42.005594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] PATH_INFO: `/v3/groups/4787ca47c646401a8181ee3e0e19e19b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:42.007833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] RBAC: Authorizing `identity:get_group(group_id=4787ca47c646401a8181ee3e0e19e19b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:42.011333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:42.013440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 205/412] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:41 2022] GET /identity/v3/groups/4787ca47c646401a8181ee3e0e19e19b => generated 229 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:42.019629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:42.053738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:42.062166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46750f01-9395-48ed-b5cb-63f3b92ef1a9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:42.140093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:42.140808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:42.141081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:42.141347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/groups/ca513b2088c2432395c23921087b2075/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:42.145461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] RBAC: Authorizing `identity:create_grant(project_id=16f3442e76ed418d936b9a5eb6f19e81, group_id=ca513b2088c2432395c23921087b2075, role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:42.147014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:42.175174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 208/413] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 13 17:47:41 2022] PUT /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81/groups/ca513b2088c2432395c23921087b2075/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:42.205520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea1211-93bd-4c66-bfb4-08d2f3967e3a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:42.257080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6aec88-7e26-432c-bc77-21dcc2e142f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:42.257728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aec88-7e26-432c-bc77-21dcc2e142f8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:42.258003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aec88-7e26-432c-bc77-21dcc2e142f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:42.258272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6aec88-7e26-432c-bc77-21dcc2e142f8 None admin] PATH_INFO: `/v3/projects/c326683f56fe42d09b126fc368517cb3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:42.259486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aec88-7e26-432c-bc77-21dcc2e142f8 None admin] RBAC: Authorizing `identity:get_project(project_id=c326683f56fe42d09b126fc368517cb3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:42.261061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6aec88-7e26-432c-bc77-21dcc2e142f8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:42.263566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 206/414] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:42 2022] GET /identity/v3/projects/c326683f56fe42d09b126fc368517cb3 => generated 344 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:42.269745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:42.403928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc40f64-a2dd-432f-bdb7-82fafd889e00 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:42.447839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d3717-c892-4160-a79e-e7783c95afad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:42.448576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d3717-c892-4160-a79e-e7783c95afad None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:42.448855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d3717-c892-4160-a79e-e7783c95afad None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:42.449132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d3717-c892-4160-a79e-e7783c95afad None admin] PATH_INFO: `/v3/projects/c326683f56fe42d09b126fc368517cb3/groups/4787ca47c646401a8181ee3e0e19e19b/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:42.451233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d3717-c892-4160-a79e-e7783c95afad None admin] RBAC: Authorizing `identity:create_grant(project_id=c326683f56fe42d09b126fc368517cb3, group_id=4787ca47c646401a8181ee3e0e19e19b, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:42.452857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d3717-c892-4160-a79e-e7783c95afad None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:42.480942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 209/415] 10.222.0.43 () {58 vars in 1442 bytes} [Tue Sep 13 17:47:42 2022] PUT /identity/v3/projects/c326683f56fe42d09b126fc368517cb3/groups/4787ca47c646401a8181ee3e0e19e19b/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:47:42.818673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aa5ed-0e60-4c9d-965c-af6784ac453f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:42.819239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aa5ed-0e60-4c9d-965c-af6784ac453f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:42.819239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1aa5ed-0e60-4c9d-965c-af6784ac453f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:42.821013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 207/416] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:42.827078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90517fb6-0733-4fdd-9f1a-6b6a28040fac None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:42.827606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90517fb6-0733-4fdd-9f1a-6b6a28040fac None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:42.827606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90517fb6-0733-4fdd-9f1a-6b6a28040fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:42.849381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-90517fb6-0733-4fdd-9f1a-6b6a28040fac None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:42.850676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90517fb6-0733-4fdd-9f1a-6b6a28040fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:42.872092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:42.945527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 210/417] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:43.325198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1aa5ed-0e60-4c9d-965c-af6784ac453f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:43.379531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:43.474355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1aa5ed-0e60-4c9d-965c-af6784ac453f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:43.524091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:43.525022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:43.525442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:43.525887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:43.542551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:43.547787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:43.548908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:43.549908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:43.552025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 208/418] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:43.558498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90517fb6-0733-4fdd-9f1a-6b6a28040fac None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:43.720743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90517fb6-0733-4fdd-9f1a-6b6a28040fac None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:43.759327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1eaf7-c501-439b-abaa-c9a09ab18470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:43.759951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1eaf7-c501-439b-abaa-c9a09ab18470 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:43.760222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1eaf7-c501-439b-abaa-c9a09ab18470 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:43.760515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1eaf7-c501-439b-abaa-c9a09ab18470 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:43.761207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1eaf7-c501-439b-abaa-c9a09ab18470 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:43.761502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1eaf7-c501-439b-abaa-c9a09ab18470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:43.763434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1eaf7-c501-439b-abaa-c9a09ab18470 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:43.774342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 211/419] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:43.786344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:43.912999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:43.923326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd262bc-113d-4abc-a750-9affc660bb0d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:43.967449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327caa9c-3125-40c1-ab83-c085e996a911 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:43.967989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327caa9c-3125-40c1-ab83-c085e996a911 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:43.968203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327caa9c-3125-40c1-ab83-c085e996a911 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:43.968882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:43.969679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:43.970086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:43.970514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:43.971475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 212/420] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:43.977304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec40d5-f5fd-495c-bddb-02a88a70a915 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:43.977748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec40d5-f5fd-495c-bddb-02a88a70a915 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:43.978156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec40d5-f5fd-495c-bddb-02a88a70a915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:43.999423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-19ec40d5-f5fd-495c-bddb-02a88a70a915 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:44.000860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-19ec40d5-f5fd-495c-bddb-02a88a70a915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:44.004501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:44.102709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:44.105436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:44.145397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 213/421] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:44.146941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc18ff-9bcf-4e78-a496-1909a452d0b8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:44.147225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc18ff-9bcf-4e78-a496-1909a452d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:44.147497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc18ff-9bcf-4e78-a496-1909a452d0b8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:44.149077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 214/422] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:44.166707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71ad61-44d0-4aa9-8d34-29724ab7f421 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:44.167033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71ad61-44d0-4aa9-8d34-29724ab7f421 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:44.167252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71ad61-44d0-4aa9-8d34-29724ab7f421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:44.199451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-db71ad61-44d0-4aa9-8d34-29724ab7f421 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:44.200740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-db71ad61-44d0-4aa9-8d34-29724ab7f421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:44.286214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:44.301839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 209/423] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:44.308039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:44.346124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 215/424] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:44.473191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71ad61-44d0-4aa9-8d34-29724ab7f421 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:44.487382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d1c5e-4206-4744-a026-66071f558d88 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:44.564145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:44.565145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:44.565615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:44.566078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:44.569618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:44.572374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:44.582726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 210/425] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:44 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:44.584330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:44.624517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:44.732930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71ad61-44d0-4aa9-8d34-29724ab7f421 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:44.754493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:44.799857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deeca8a-dbad-4df5-bc4d-cbb948dfbdfb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:44.804294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:44.805689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:44.806368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:44.807031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:44.828216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:44.839508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:44.841563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 17:47:44.843314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:44.845005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:44.845005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:44.845005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:44.845005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:44.845780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:44.847344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:44.848900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 216/426] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:47:44.850736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:44.854430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 211/427] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:44 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:44.856581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:44.984311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbc44d-cf14-44e0-8f63-e542344fa5d8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:45.024967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:45.035009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:45.036368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:45.036870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:45.037393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:45.038387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] RBAC: Adding query filter params (name=member) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:45.038907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:45.040679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:45.051701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 212/428] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:47:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:45.054518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:45.058941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd171a-862b-49dd-bcdf-1a4613f43b74 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:45.123276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:45.135968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:45.137990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:45.138466 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:45.138913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:45.162512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:45.172661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:45.173624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:45.174408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:45.175010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 217/429] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:45.180708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:45.221783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a5aa6-6622-43be-b51f-1ce5739b0671 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:45.263793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:45.264912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:45.265384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:45.265887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:45.266852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=a5a5bade20834645af2806f86402c383) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:45.267399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:45.269532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:45.278850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 213/430] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:45 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=5c592b5b170d48b0825640b8283425de => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:45.280335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:45.354874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:45.367808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c368314-1aa8-4aec-bccf-81b1855cc1cf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:45.413315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ebe8d-e4d7-438f-b229-0f3badeff87a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:45.431071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:45.432082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:45.432576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:45.433043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:45.487217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6851e-7271-47ad-9dca-8c3bbd6d83fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:45.488775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6851e-7271-47ad-9dca-8c3bbd6d83fe None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:45.489904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6851e-7271-47ad-9dca-8c3bbd6d83fe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:45.490650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6851e-7271-47ad-9dca-8c3bbd6d83fe None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:45.492907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6851e-7271-47ad-9dca-8c3bbd6d83fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:45.494284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6851e-7271-47ad-9dca-8c3bbd6d83fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:45.497244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6851e-7271-47ad-9dca-8c3bbd6d83fe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:45.513241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 214/431] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:45.515907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af8c15-7116-4a41-8780-3b4a75e33765 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:45.516762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af8c15-7116-4a41-8780-3b4a75e33765 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:45.517492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af8c15-7116-4a41-8780-3b4a75e33765 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:45.523707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 215/432] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:45.528476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45919b-2fc8-45ab-b9a1-9f0c040586ae None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:45.528476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45919b-2fc8-45ab-b9a1-9f0c040586ae None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:45.528476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45919b-2fc8-45ab-b9a1-9f0c040586ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:45.540852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:45.563538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ee45919b-2fc8-45ab-b9a1-9f0c040586ae None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:45.565505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:45.565730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ee45919b-2fc8-45ab-b9a1-9f0c040586ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:45.568712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:45.716842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 216/433] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:45.718499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45919b-2fc8-45ab-b9a1-9f0c040586ae None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:45.798218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 218/434] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:45.807981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:45.842318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:45.943231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45919b-2fc8-45ab-b9a1-9f0c040586ae None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:45.999469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:46.016250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0836aa-7acb-4c99-b953-019b0fba58a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:46.017335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575d4c2-66cc-4cad-b2f5-4619237b45ec None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:46.018685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0836aa-7acb-4c99-b953-019b0fba58a5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.019157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0836aa-7acb-4c99-b953-019b0fba58a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.019642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0836aa-7acb-4c99-b953-019b0fba58a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.082501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081217b-6435-468a-970a-055308213703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:46.083446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081217b-6435-468a-970a-055308213703 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.083860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081217b-6435-468a-970a-055308213703 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.084368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081217b-6435-468a-970a-055308213703 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.086486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081217b-6435-468a-970a-055308213703 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:46.089423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081217b-6435-468a-970a-055308213703 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:46.092555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 219/435] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:45 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:46.094671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a94b-2574-4081-babd-d2342bf7159c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.095130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a94b-2574-4081-babd-d2342bf7159c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.095601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a94b-2574-4081-babd-d2342bf7159c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.097960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 220/436] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:46.101584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a94b-2574-4081-babd-d2342bf7159c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:46.143407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0836aa-7acb-4c99-b953-019b0fba58a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:46.144957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0836aa-7acb-4c99-b953-019b0fba58a5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:46.217442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:46.266988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a94b-2574-4081-babd-d2342bf7159c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:46.285029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 217/437] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:46.287157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42d16c-3680-4c3e-82dc-94cde9db7542 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.287627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42d16c-3680-4c3e-82dc-94cde9db7542 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.288099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42d16c-3680-4c3e-82dc-94cde9db7542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.309871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dd42d16c-3680-4c3e-82dc-94cde9db7542 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:46.311233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dd42d16c-3680-4c3e-82dc-94cde9db7542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:46.313829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:46.314701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.315096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.315479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.316962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:46.318757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:46.320772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 221/438] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:46 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:46.321756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:46.361297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:46.415716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 218/439] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:46.417754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42d16c-3680-4c3e-82dc-94cde9db7542 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:46.494150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537a867-b0d1-4a6f-b1c3-737d77a7fda0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:46.526824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:46.558596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42d16c-3680-4c3e-82dc-94cde9db7542 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:46.560192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:46.561379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.561852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.562321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.583693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:46.593572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:46.595235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:46.596998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 222/440] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:47:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:46.598564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:46.615417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:46.616510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.617237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.618027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] PATH_INFO: `/v3/projects/a5a5bade20834645af2806f86402c383` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.620757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] RBAC: Authorizing `identity:get_project(project_id=a5a5bade20834645af2806f86402c383)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:46.623306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:46.626499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 219/441] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:46 2022] GET /identity/v3/projects/a5a5bade20834645af2806f86402c383 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:46.627549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:46.751777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:46.772480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7fe90-1deb-42cb-b82d-de5b9866719a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:46.778385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708cfec-6621-43cc-9b23-be32e36d73bb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:46.815215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:46.816216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.816718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.817159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.818523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=5c592b5b170d48b0825640b8283425de, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:46.819072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:46.821923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:46.822611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:46.823273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:46.823551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:46.823822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:46.824569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:46.824885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:46.826634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:46.833132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 223/442] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:46 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=5c592b5b170d48b0825640b8283425de => generated 611 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:46.834809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:46.837871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 220/443] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:46.839102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:46.859936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:46.997197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b50cdd-672a-4a7b-906e-c7a7456c665f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.004319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276d0e1-3a34-425c-b537-dc6d81bf6622 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.040208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.041078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.041078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.041297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.053771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:47.060150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:47.061879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 221/444] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:47.063350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:47.076814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.077862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.078294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.078753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.080488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:47.082994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:47.085841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 224/445] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:46 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:47.092459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:47.118237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:47.172955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:47.218404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920bdf9-d180-4cf1-8baf-b3d2bb33641c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.229021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386097f-5671-4b5c-9b52-4d7838e2c558 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.252983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.253557 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.253841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.254110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.254789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:47.255089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:47.256603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:47.265424 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 225/446] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:47.265424 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:47.279089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.279844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.280281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.280667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.357300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:47.358959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:47.389342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:47.420204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae2457-0e0d-4b6b-bbcf-d51c2b14cbf5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.431395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:47.463589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.464354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.464795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.465228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.466121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=a5a5bade20834645af2806f86402c383) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:47.466580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:47.468246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:47.475202 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 226/447] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=a5a5bade20834645af2806f86402c383&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:47.476225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:47.495139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 222/448] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:47.501932 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:47.601702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b283de-fc84-4bc3-a029-3b1524abd469 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.641503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.642135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.642484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.642917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.657484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:47.672183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144274f-4c6f-41e5-9e1e-29653fb5831e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.729663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.731209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.731893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.733634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] PATH_INFO: `/v3/projects/c326683f56fe42d09b126fc368517cb3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.735463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:47.736390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] RBAC: Authorizing `identity:get_project(project_id=c326683f56fe42d09b126fc368517cb3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:47.737064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:47.740067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:47.745905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 223/449] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/projects/c326683f56fe42d09b126fc368517cb3 => generated 344 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:47.753840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:47.814843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:47.867257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 227/450] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:47:47.876780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:47.908250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c05b1-60df-47ea-95c6-f49e97700b1f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:47.953543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:47.963629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:47.964591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:47.965079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:47.968475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] PATH_INFO: `/v3/groups/4787ca47c646401a8181ee3e0e19e19b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:47.968475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] RBAC: Authorizing `identity:get_group(group_id=4787ca47c646401a8181ee3e0e19e19b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:47.969742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:47.972705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 224/451] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/groups/4787ca47c646401a8181ee3e0e19e19b => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:47.979244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:48.069385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0474f-1fea-4d46-a6c0-eaf94af8e2a2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:48.124152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:48.125299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:48.125734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:48.127992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] PATH_INFO: `/v3/projects/16f3442e76ed418d936b9a5eb6f19e81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:48.128162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] RBAC: Authorizing `identity:get_project(project_id=16f3442e76ed418d936b9a5eb6f19e81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:48.132691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:48.135481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 228/452] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/projects/16f3442e76ed418d936b9a5eb6f19e81 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:48.143406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:48.161654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9475bc-37cb-4dcf-b38d-046294c2f3ff None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:48.165975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:48.213756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:48.214371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:48.214644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:48.214906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:48.215654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] RBAC: Adding query filter params (group.id=4787ca47c646401a8181ee3e0e19e19b, role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=c326683f56fe42d09b126fc368517cb3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:48.215969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:48.217479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:48.225326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 225/453] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:47 2022] GET /identity/v3/role_assignments?group.id=4787ca47c646401a8181ee3e0e19e19b&scope.project.id=c326683f56fe42d09b126fc368517cb3&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 611 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:48.327057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcd03c-50fa-4b64-bf1a-e46ecaf604e2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:48.347097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:48.379954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12c6b0-2b79-4e17-84d2-40b996d48e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:48.380868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c6b0-2b79-4e17-84d2-40b996d48e8d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:48.381282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c6b0-2b79-4e17-84d2-40b996d48e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:48.381640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c6b0-2b79-4e17-84d2-40b996d48e8d None admin] PATH_INFO: `/v3/groups/ca513b2088c2432395c23921087b2075` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:48.383115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12c6b0-2b79-4e17-84d2-40b996d48e8d None admin] RBAC: Authorizing `identity:get_group(group_id=ca513b2088c2432395c23921087b2075)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:48.384700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12c6b0-2b79-4e17-84d2-40b996d48e8d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:48.386443 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 229/454] 10.222.0.43 () {56 vars in 1180 bytes} [Tue Sep 13 17:47:48 2022] GET /identity/v3/groups/ca513b2088c2432395c23921087b2075 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:47:48.393424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:48.520325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03b068-180b-4abf-944c-a3f8802064bd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:48.557571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68067f8c-f015-4ed6-8902-ecc133fc4347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:48.558319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067f8c-f015-4ed6-8902-ecc133fc4347 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:48.558678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067f8c-f015-4ed6-8902-ecc133fc4347 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:48.559031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68067f8c-f015-4ed6-8902-ecc133fc4347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:48.559854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68067f8c-f015-4ed6-8902-ecc133fc4347 None admin] RBAC: Adding query filter params (group.id=ca513b2088c2432395c23921087b2075, role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=16f3442e76ed418d936b9a5eb6f19e81) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:48.560246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68067f8c-f015-4ed6-8902-ecc133fc4347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:48.561833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68067f8c-f015-4ed6-8902-ecc133fc4347 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:48.568810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 226/455] 10.222.0.43 () {56 vars in 1376 bytes} [Tue Sep 13 17:47:48 2022] GET /identity/v3/role_assignments?group.id=ca513b2088c2432395c23921087b2075&scope.project.id=16f3442e76ed418d936b9a5eb6f19e81&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:51.551113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b701ea-fbc9-4656-8de9-c37c636c0afd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:51.551674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b701ea-fbc9-4656-8de9-c37c636c0afd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:51.551944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b701ea-fbc9-4656-8de9-c37c636c0afd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:51.553589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 230/456] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:51.562009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fdfec-0310-4c5a-b994-585a6f53d06b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:51.562510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fdfec-0310-4c5a-b994-585a6f53d06b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:51.562754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fdfec-0310-4c5a-b994-585a6f53d06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:51.589622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-807fdfec-0310-4c5a-b994-585a6f53d06b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:51.590860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-807fdfec-0310-4c5a-b994-585a6f53d06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:51.623642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:51.687024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 227/457] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:52.003685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539967d-096b-4b3b-b5e2-84099e72d741 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.004335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539967d-096b-4b3b-b5e2-84099e72d741 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.004882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539967d-096b-4b3b-b5e2-84099e72d741 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.006887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 231/458] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:52.012840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088346b-2e8a-4076-982e-b1ebcef052b8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.013154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088346b-2e8a-4076-982e-b1ebcef052b8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.013394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088346b-2e8a-4076-982e-b1ebcef052b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.025441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539967d-096b-4b3b-b5e2-84099e72d741 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:52.045272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4088346b-2e8a-4076-982e-b1ebcef052b8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:52.046481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4088346b-2e8a-4076-982e-b1ebcef052b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:52.171222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 228/459] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:52.173473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eb1a2-ab79-4c3f-92fb-8c870804ffda None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.173966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eb1a2-ab79-4c3f-92fb-8c870804ffda None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.174406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26eb1a2-ab79-4c3f-92fb-8c870804ffda None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.183740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 229/460] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:52.186023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5d0ff-0069-453d-ade9-0a3e15e7daf7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.186513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5d0ff-0069-453d-ade9-0a3e15e7daf7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.186991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5d0ff-0069-453d-ade9-0a3e15e7daf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.187508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539967d-096b-4b3b-b5e2-84099e72d741 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:52.217073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d9f5d0ff-0069-453d-ade9-0a3e15e7daf7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:52.219108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d9f5d0ff-0069-453d-ade9-0a3e15e7daf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:52.226469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:52.227123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.227397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.227667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.239870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:52.241500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:52.261146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:52.262470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:52.263322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:52.263322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:52.263322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:52.263322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 232/461] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:52.270495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:52.351129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:52.375475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 230/462] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:52.476994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba764-4708-4337-81c7-0aa09dd05950 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:52.491274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c9098-999d-4ab2-b8f3-b0b0d23514ae None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.491597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c9098-999d-4ab2-b8f3-b0b0d23514ae None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.491823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c9098-999d-4ab2-b8f3-b0b0d23514ae None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.493575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 231/463] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:52.502790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb4e70-6114-445e-b58e-ce2e6f25e86f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.503105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb4e70-6114-445e-b58e-ce2e6f25e86f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.503347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb4e70-6114-445e-b58e-ce2e6f25e86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.524748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:52.525365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.525633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.525907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.526566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:52.526862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:52.528296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:52.542032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c6eb4e70-6114-445e-b58e-ce2e6f25e86f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:52.543813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c6eb4e70-6114-445e-b58e-ce2e6f25e86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:52.548517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 233/464] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:52.550228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:52.575871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:52.643558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 232/465] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:52.643558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb4e70-6114-445e-b58e-ce2e6f25e86f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:52.694552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:52.726773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43eeea5-453e-4a8d-a9a0-231c4b8fc67a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:52.771084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:52.771712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.772000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.772276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.783856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:52.785392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:52.790221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb4e70-6114-445e-b58e-ce2e6f25e86f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:52.803699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:52.804932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:52.806157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:52.806157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:52.806157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:52.806157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 234/466] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:52.806157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:52.833428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:52.834563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:52.835099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:52.835561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:52.836528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:52.838188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:52.870502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 233/467] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:52 2022] POST /identity/v3/users => generated 246 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:52.872017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:52.886779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:52.947004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9decc96-5b08-49c7-adb6-9abba859eb1b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:53.009583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:53.010429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.010605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.010873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.025206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecbbac-8d1a-48e8-9b93-0c59587a287d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:53.028389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:53.030796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:53.033479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:53.058432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:53.059360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:53.060213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:53.060213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:53.060213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:53.061388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 235/468] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:53.063020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:53.075909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:53.076951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.077425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.077872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.078855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:53.079348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:53.081863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:53.098808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 234/469] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:53.100171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:53.230157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:53.239800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5015e77-8d52-42a0-9831-335b4d0e3125 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:53.292833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ded3f-d6cd-4d3a-b356-911f450fa99c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:53.304440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:53.305595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.306087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.306875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.326977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:53.329656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:53.344415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9796d4-79fa-47a1-9d43-2b00a55cf4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:53.345066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9796d4-79fa-47a1-9d43-2b00a55cf4ac None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.345884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9796d4-79fa-47a1-9d43-2b00a55cf4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.345884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9796d4-79fa-47a1-9d43-2b00a55cf4ac None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.346336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9796d4-79fa-47a1-9d43-2b00a55cf4ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:53.346685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9796d4-79fa-47a1-9d43-2b00a55cf4ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:53.348107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9796d4-79fa-47a1-9d43-2b00a55cf4ac None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:53.357813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:53.359553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:53.364128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:53.364128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:53.364128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:53.364128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 236/470] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:53.364128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:53.369707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 235/471] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:53.370567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384be0c0-01a4-4e45-8163-bd93229f8900 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.370851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384be0c0-01a4-4e45-8163-bd93229f8900 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.371129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384be0c0-01a4-4e45-8163-bd93229f8900 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.376238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 236/472] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:53.376238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384be0c0-01a4-4e45-8163-bd93229f8900 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:53.432008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:53.499488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744c2d2-bc16-4afd-ab53-92c70a55b52c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:53.546773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:53.548325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.549242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.549919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.552190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:53.556125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:53.573825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384be0c0-01a4-4e45-8163-bd93229f8900 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:53.601711 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 237/473] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:53 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:53.603643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:53.635911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:53.652118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42f275-863e-4c34-ac72-dbdeea01d8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:53.653252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f275-863e-4c34-ac72-dbdeea01d8ac None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.653838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f275-863e-4c34-ac72-dbdeea01d8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.655390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f275-863e-4c34-ac72-dbdeea01d8ac None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.656826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f275-863e-4c34-ac72-dbdeea01d8ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:53.657406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f275-863e-4c34-ac72-dbdeea01d8ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:53.660313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f275-863e-4c34-ac72-dbdeea01d8ac None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:53.697098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 237/474] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:53.700562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010ec3-d096-4edc-a74a-86bc624ee02c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.701185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010ec3-d096-4edc-a74a-86bc624ee02c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.701740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d010ec3-d096-4edc-a74a-86bc624ee02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.738317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8d010ec3-d096-4edc-a74a-86bc624ee02c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:53.740408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d010ec3-d096-4edc-a74a-86bc624ee02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:53.834872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243e14d-5aec-4ea9-adc2-bb4c8d99e330 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:53.839066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:53.888568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 238/475] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:53.888568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d010ec3-d096-4edc-a74a-86bc624ee02c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:53.945132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:53.946109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:53.946561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:53.947008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:53.948077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:53.950829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:53.996286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 238/476] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:53 2022] POST /identity/v3/users => generated 248 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:54.077551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d010ec3-d096-4edc-a74a-86bc624ee02c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:54.114151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:54.147629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:54.148463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:54.148905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:54.149310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:54.150343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:54.152836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:54.195693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 239/477] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:47:53 2022] POST /identity/v3/users => generated 249 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:47:54.485968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:54.679380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c646-09a3-40a6-be18-7de3c89675a5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:54.749568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:54.750584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:54.751063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:54.751529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:54.752863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:54.755404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:54.782553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 17:47:54.782553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 17:47:54.782553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [parameters: {'id': 'dfa3c180c56545dab18c078696894388', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 17:47:54.782553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Sep 13 17:47:54.818563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 17:47:54.820285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 17:47:54.822298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 17:47:54.824006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 17:47:54.827922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 17:47:54.836238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'id': 'dfa3c180c56545dab18c078696894388', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:54.839166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 17:47:54.842334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:54.850191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 239/478] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 13 17:47:54 2022] POST /identity/v3/roles => generated 144 bytes in 336 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:54.850424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:55.006470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def80fa-bd40-4216-9887-53fdb1bd3f1a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:55.057095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb71a8-dc1f-45db-b3e7-a1d241847e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:55.057882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb71a8-dc1f-45db-b3e7-a1d241847e42 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:55.058163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb71a8-dc1f-45db-b3e7-a1d241847e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:55.058438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb71a8-dc1f-45db-b3e7-a1d241847e42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:55.076450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb71a8-dc1f-45db-b3e7-a1d241847e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-81cb71a8-dc1f-45db-b3e7-a1d241847e42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:55.081979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:55.085416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:55.088597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 240/479] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:47:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:55.096463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:55.174067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:55.270206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dac2fa-77a2-4b31-b839-f28a662b01e4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:55.312196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41565f8-5644-41a3-9728-206929f1e2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:55.312861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41565f8-5644-41a3-9728-206929f1e2ae None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:55.313162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41565f8-5644-41a3-9728-206929f1e2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:55.313392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41565f8-5644-41a3-9728-206929f1e2ae None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:55.314125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41565f8-5644-41a3-9728-206929f1e2ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:55.314462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41565f8-5644-41a3-9728-206929f1e2ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:55.315898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41565f8-5644-41a3-9728-206929f1e2ae None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:55.331982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 240/480] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:55.513596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cb810-bbe1-4722-9381-0485965165e3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:55.514090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cb810-bbe1-4722-9381-0485965165e3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:55.514339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cb810-bbe1-4722-9381-0485965165e3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:55.515930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 241/481] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:55.522197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b21a5-2853-4098-a9a3-8f79192f54fe None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:55.522586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b21a5-2853-4098-a9a3-8f79192f54fe None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:55.522798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b21a5-2853-4098-a9a3-8f79192f54fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:55.543170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8f0b21a5-2853-4098-a9a3-8f79192f54fe None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:55.544479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8f0b21a5-2853-4098-a9a3-8f79192f54fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:55.648732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 241/482] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:55.930094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cb810-bbe1-4722-9381-0485965165e3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:56.005197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:56.131501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cb810-bbe1-4722-9381-0485965165e3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:56.202361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:56.202912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:56.203219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:56.203450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:56.226223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:56.236754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:56.240116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:56.242653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 242/483] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:47:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:47:56.252473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b21a5-2853-4098-a9a3-8f79192f54fe None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:56.282491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:56.452207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b21a5-2853-4098-a9a3-8f79192f54fe None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:56.533602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:56.534567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:56.534984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:56.535397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:56.536535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:56.537002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:56.539437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:56.558923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 242/484] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:47:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:56.565147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:56.744158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:56.755091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074ff16-7aa8-4fc3-ad15-b18663c063aa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:56.814063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:56.815016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:56.815490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:56.815923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:56.835727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:56.838367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:56.864930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:56.865997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:56.866863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:56.866863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:56.866863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:56.867464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 243/485] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:47:56.874485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:57.026805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:57.068510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379f022-28ba-4beb-a5dd-b09e4f365b67 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:57.133335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:57.134339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:57.134827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:57.135292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:57.136528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:57.137490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:57.139655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:57.164214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 243/486] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:57.170692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:57.370710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db138d11-9ab7-46c4-8d0d-5a0b1ad7f7cb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:57.437474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84bb94-d1c6-43bd-a935-b662b5b50c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:57.438598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84bb94-d1c6-43bd-a935-b662b5b50c18 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:57.439081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84bb94-d1c6-43bd-a935-b662b5b50c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:57.439546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84bb94-d1c6-43bd-a935-b662b5b50c18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:57.455957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84bb94-d1c6-43bd-a935-b662b5b50c18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:57.462738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84bb94-d1c6-43bd-a935-b662b5b50c18 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9d84bb94-d1c6-43bd-a935-b662b5b50c18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:47:57.489562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:57.490613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:47:57.491511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:57.492044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 244/487] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:47:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:57.498039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:57.660286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846baca-e3a9-43bc-bd8b-9b0735ac7bd8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:57.725947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5dcac-3cc8-4e3c-8961-aea3d611fb97 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:57.726359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:57.726482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5dcac-3cc8-4e3c-8961-aea3d611fb97 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:57.726862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5dcac-3cc8-4e3c-8961-aea3d611fb97 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:57.726967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:57.727250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:57.727519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:57.728278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:57.728619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:57.730148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:57.738068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 245/488] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:57 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:57.745227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:57.748366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b080916-fc71-45b5-b01c-e63fa324102e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:57.748988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b080916-fc71-45b5-b01c-e63fa324102e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:57.749452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b080916-fc71-45b5-b01c-e63fa324102e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:57.776865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:57.784905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3b080916-fc71-45b5-b01c-e63fa324102e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:57.794557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3b080916-fc71-45b5-b01c-e63fa324102e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:57.812065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 244/489] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:47:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:57.818124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:57.919334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 246/490] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:57.971710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924f262-d95b-4cd1-bc60-1648dda0d8f8 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:57.971999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924f262-d95b-4cd1-bc60-1648dda0d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:57.972324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924f262-d95b-4cd1-bc60-1648dda0d8f8 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:57.973872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 247/491] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:47:57.979434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d04d-574c-44a8-aa74-1d21e40aa358 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:57.979799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d04d-574c-44a8-aa74-1d21e40aa358 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:57.979970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d04d-574c-44a8-aa74-1d21e40aa358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:57.999730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f418d04d-574c-44a8-aa74-1d21e40aa358 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:58.000982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f418d04d-574c-44a8-aa74-1d21e40aa358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:58.010156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9a46-e5ca-47b4-8f6b-a84ecef4d767 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:58.055097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:58.055397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.055653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.055925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.097770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 248/492] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:47:58.099424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128dbb-69c5-4626-b339-1a96ea9a215b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.099911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128dbb-69c5-4626-b339-1a96ea9a215b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.100324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128dbb-69c5-4626-b339-1a96ea9a215b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.101853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 249/493] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:58.108301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a8470-ce43-410d-850b-f60ece1f4912 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.108813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a8470-ce43-410d-850b-f60ece1f4912 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.109224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a8470-ce43-410d-850b-f60ece1f4912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.156000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:58.157778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3f7a8470-ce43-410d-850b-f60ece1f4912 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:58.159077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3f7a8470-ce43-410d-850b-f60ece1f4912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:58.164311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:58.166325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:58.289186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 245/494] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 17:47:58.290550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:58.296381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:58.314526 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 250/495] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:58.316718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a8470-ce43-410d-850b-f60ece1f4912 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:58.478579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:58.492691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6b8b-6056-4bfa-92f0-8156a1e239c9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:58.500858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a8470-ce43-410d-850b-f60ece1f4912 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:58.542742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:58.543390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.543574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.543830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.547641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:58.548415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.548816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.549187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.555988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:58.557557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:58.565165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:58.572011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:58.573147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:58.573941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 246/496] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:47:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:58.574594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:58.586454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:58.587454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:58.588877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:58.588877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:58.588877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:58.592662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 251/497] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:58.593731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:58.653846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:58.709618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b243-8a41-4e5f-9628-5378adc83a10 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:58.726022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5796204-53d2-4bd0-8ec0-6eb416e3df53 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:58.758971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:58.760086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.760747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.761254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.774891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:58.775491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.775768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.776041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.776609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:58.776809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:58.777105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:58.778589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:58.783031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:58.784013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:58.785435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:58.793043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 247/498] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:47:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:58.793728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 252/499] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:47:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:58.794801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:58.794969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:58.815532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:58.917271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ed393-7938-4ac2-8091-57b9d88bd47b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:58.928367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f78ef-6eb4-4aee-bd1b-3e57d0e531f6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:58.929726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:58.957328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:58.957971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.958248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.958505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.971424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:58.972927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:58.973112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:58.973956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:58.974317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:58.974599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:58.975888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:58.976401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:58.978936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:58.988626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:58.990120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:58.991775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:58.991775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:58.991775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:58.991775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 253/500] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:58.991775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.007797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 248/501] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:59.007797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.105975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:59.112131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b25bdf-7854-408c-9f72-c2f19a48a536 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.137387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8730ffd-a034-415e-97a5-d9e12518de05 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.153746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198a86c-b315-46a5-b20b-649e3ebcd8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.154987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198a86c-b315-46a5-b20b-649e3ebcd8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.155371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198a86c-b315-46a5-b20b-649e3ebcd8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.155729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198a86c-b315-46a5-b20b-649e3ebcd8e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.156563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198a86c-b315-46a5-b20b-649e3ebcd8e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:59.156955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198a86c-b315-46a5-b20b-649e3ebcd8e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:59.158550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198a86c-b315-46a5-b20b-649e3ebcd8e0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:59.168276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 254/502] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:47:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:59.169854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672261-b8b0-4f2e-9c1f-67e3b245ff4d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.170143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672261-b8b0-4f2e-9c1f-67e3b245ff4d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.170410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672261-b8b0-4f2e-9c1f-67e3b245ff4d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.171655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 255/503] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:47:59.174700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b672261-b8b0-4f2e-9c1f-67e3b245ff4d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.178933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.179582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.179858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.180126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.191916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:47:59.197747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:59.199679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:47:59.201529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:59.203955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 249/504] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:59.204913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.209648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:59.305633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b672261-b8b0-4f2e-9c1f-67e3b245ff4d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.325294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f56c-62ba-4830-9799-4d314a0698ec None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.347185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.347919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.348316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.348736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.349843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:59.350326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:59.352613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:59.365973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edbf43-6c19-4eab-8b88-5ae875e50b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.366554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edbf43-6c19-4eab-8b88-5ae875e50b48 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.366871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edbf43-6c19-4eab-8b88-5ae875e50b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.367193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edbf43-6c19-4eab-8b88-5ae875e50b48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.367520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 256/505] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:59.368401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.370175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:59.377461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edbf43-6c19-4eab-8b88-5ae875e50b48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:59.380514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edbf43-6c19-4eab-8b88-5ae875e50b48 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c2edbf43-6c19-4eab-8b88-5ae875e50b48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:47:59.398008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:59.399914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 250/506] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aca10-1689-48bc-8314-b3789407a7e1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.401856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aca10-1689-48bc-8314-b3789407a7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.403146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32aca10-1689-48bc-8314-b3789407a7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.429062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b32aca10-1689-48bc-8314-b3789407a7e1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:47:59.430612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b32aca10-1689-48bc-8314-b3789407a7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:47:59.503335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:59.525069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713efe-43f0-4cc4-b1a7-aa2c7e30c9c4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.528467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 251/507] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:47:59.529647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32aca10-1689-48bc-8314-b3789407a7e1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.605689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.606809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.607294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.607771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.630379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:59.634052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:47:59.664658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:59.665652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:47:59.666501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:47:59.666501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:47:59.666501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:59.666785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 257/508] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:59.668127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.681607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32aca10-1689-48bc-8314-b3789407a7e1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.716565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:59.733201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03274999-248a-486c-be77-b1d460578c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.734210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03274999-248a-486c-be77-b1d460578c48 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.734665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03274999-248a-486c-be77-b1d460578c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.735140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03274999-248a-486c-be77-b1d460578c48 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.736990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03274999-248a-486c-be77-b1d460578c48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:59.736990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03274999-248a-486c-be77-b1d460578c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:59.739311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03274999-248a-486c-be77-b1d460578c48 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:59.760497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 252/509] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:47:59.760497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03274999-248a-486c-be77-b1d460578c48 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.829323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4e50-2c7e-4573-9b73-b7b7de166b87 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.892072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.893443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.894505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.895095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.900269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:47:59.908547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03274999-248a-486c-be77-b1d460578c48 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:47:59.910803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:59.913358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:47:59.934790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:47:59.936754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:47:59.938781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:47:59.943711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 258/510] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:47:59.945283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:47:59.974221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:47:59.975402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:47:59.975890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:47:59.976357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:47:59.977674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:47:59.978210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:47:59.980902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.014067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 253/511] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:00.015418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.080077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27829eb1-a615-40bf-b6ee-ad6fa4e2ac69 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.082720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:00.125157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.125833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.126167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.126494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.127249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:00.127736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.129381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.145168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 259/512] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:00.146832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.157446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6806c-43d6-4491-9366-bcd7f1f698c1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.201374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.202264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.202708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.203159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.218593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.221178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.222288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:00.241375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:00.243344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:00.245266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:00.245266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:00.245266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:00.245266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 254/513] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:00.245266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.302935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc88cb-86b6-4633-93a4-500c0f1bd58c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.355064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.356116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.356116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.356361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.356961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:00.357263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.358735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.384001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:00.391167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0d41-4d4d-48e9-992a-4b2f6a026072 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.398598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 260/514] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:00.399374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.454042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.455131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.455629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.456104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.475763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.478584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:00.503896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:00.505922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:00.509595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:00.509595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:00.509595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:00.509595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 255/515] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:00.509595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.513444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ace3f-f832-4e5a-8a72-5ad46df336d2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.531163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:00.553521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.554142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.554419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.554690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.555425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=90f40fe8f20f4a5bbd201debda6a97a5) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:00.555742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.557291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.563199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 261/516] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/role_assignments?user.id=90f40fe8f20f4a5bbd201debda6a97a5&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:00.564042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.671831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b46637-90cd-4b86-93ff-48e7cefa7252 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.735456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a3110-7290-43be-b9a1-38a53fd4dce9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.735759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.736997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.737579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.738148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.752849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.755710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.774767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.775493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.775781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.776051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.776745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:00.777048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.778489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:00.781627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:00.782571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:00.783457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 256/517] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:00.784925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.806477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 262/518] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:00.806477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:00.808345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:00.856484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:00.941267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27176-660d-45b0-9f74-e396a95f03c5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.970237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b6337-9d39-4110-8d7d-bb0a0497590a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:00.993103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:00.993906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:00.994308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:00.994709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:00.995677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:00.996099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:00.998126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:01.014015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 263/519] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:01.015375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:01.035351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:01.036376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:01.036897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:01.037423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:01.152753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:01.155567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:01.164033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:01.169522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05bfff-8046-4a5c-875f-aad689ce97ae None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:01.178010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:01.215816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:01.216576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:01.216978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:01.217382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:01.218258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:01.218702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:01.221511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:01.251182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 264/520] 10.222.0.43 () {56 vars in 1139 bytes} [Tue Sep 13 17:48:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:01.251182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:01.323319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 257/521] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:01.323319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:01.439313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8f095-ed43-45e5-985a-1cf7efd0a0e9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:01.457251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e12bb-f6f8-4f90-ac1e-5442c1022243 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:01.472276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:01.500922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:01.501928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:01.502340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:01.502782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:01.508231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:01.509284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:01.509775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:01.510250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:01.511500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:01.513099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:01.514598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:01.514760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:01.528406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:01.530395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 17:48:01.532098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:01.536967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 265/522] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:01.538593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:01.556521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 258/523] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:48:01 2022] POST /identity/v3/users => generated 247 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:01.558638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:01.580751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:01.745630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605f98c-f0a7-475a-b94f-ec5d3b610aeb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:01.788806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160265a6-69d7-4e1d-89b7-483d840e9235 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:01.810365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:01.811413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:01.811700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:01.811984 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:01.855811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:01.856871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:01.857345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:01.857785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:01.874636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:01.877317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:01.900547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:01.902445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:01.904095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:01.904095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:01.904095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:01.904095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 259/524] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:01.904095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:01.910345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:01.913140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:01.933018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:01.949543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:02.035266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f243cd-d21e-4338-97dd-899e65f96ff7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.077133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:02.077766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:02.078047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:02.078322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:02.079170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:02.079479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:02.081150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:02.102470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 260/525] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 13 17:48:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:02.103453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:02.124850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 266/526] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:02.126482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:02.210757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:02.247085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2b98-d671-4f7a-9104-676c2368e971 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.299745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:02.300611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:02.300898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:02.301172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:02.301841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:02.302147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:02.304093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:02.319411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd107cbc-6c0a-452f-92cf-3f4ce082a0a4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.320132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 261/527] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:02.321092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:02.351296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:02.392016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:02.393043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:02.393331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:02.393605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:02.458656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c0b30-41c9-4d6d-a925-fd4626ebf97a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.501072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:02.502650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:02.519453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:02.520591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:02.521169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:02.521797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:02.527059 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:02.543373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:02.546021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:02.575764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:02.577437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:02.579235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:02.579235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:02.579235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:02.581261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 262/528] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:02.584824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:02.624493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 267/529] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:02.625392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:02.647493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:02.727869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ed01-f825-4de7-a380-d581b0c5993a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.761489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2bc44-08af-4dd8-8aff-e47a916e4f5c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.767483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:02.768581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:02.769070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:02.769612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:02.778163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:02.779952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:02.793517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:02.795031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:02.796022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:02.798084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 263/530] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:02.799151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:02.800950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:02.801579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:02.801859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:02.802134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:02.802854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:02.803498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:02.805368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:02.819352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 268/531] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:02.820418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:02.859072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:02.930860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405b9e4-915b-4650-a59a-9d5542c9073c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.938057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:02.977276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:02.978135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:02.978707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:02.978991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:02.987173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd299e-cb4a-4a7c-a1a6-2bf20375a84a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:02.997622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:02.999129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:03.020194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:03.022158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:03.024341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:03.024341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:03.024341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:03.024341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 264/532] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:03.024341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:03.059625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.061629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.062629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.064065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.065776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:03.066546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.069653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.108224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 269/533] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:03.111092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:03.161212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:03.166062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a555ab3-976a-4d09-a849-58e793cca06a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:03.221950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.222928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.223350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.223750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.233139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.235669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:03.248710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:03.249626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:03.250467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 265/534] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:03.250923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:03.275709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad8532-0ae5-4f51-bf90-d7bb2480f758 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:03.322572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82395634-4851-498b-aafc-5072c06fbf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.323572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82395634-4851-498b-aafc-5072c06fbf73 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.324035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82395634-4851-498b-aafc-5072c06fbf73 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.324326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82395634-4851-498b-aafc-5072c06fbf73 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.325030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82395634-4851-498b-aafc-5072c06fbf73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:03.325339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82395634-4851-498b-aafc-5072c06fbf73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.326985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82395634-4851-498b-aafc-5072c06fbf73 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.330654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:03.342238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 270/535] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:03.343920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbe650-fb04-434f-bfdd-26203e4bc44e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.344419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbe650-fb04-434f-bfdd-26203e4bc44e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.344928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbe650-fb04-434f-bfdd-26203e4bc44e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.346515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 271/536] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:03.347613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbe650-fb04-434f-bfdd-26203e4bc44e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:03.406055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429031b-f5b9-4bf3-8685-8a8429124b00 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:03.448610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.449554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.449554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.449879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.450599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=6c9c5205f9344e8c8fcc85c80d6f504b) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:03.450755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.452228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.468463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 266/537] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/role_assignments?user.id=6c9c5205f9344e8c8fcc85c80d6f504b&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:03.468463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:03.510115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:03.517059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbe650-fb04-434f-bfdd-26203e4bc44e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:03.558010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ad69d-0e05-4208-8017-ca7b88133229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.558801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ad69d-0e05-4208-8017-ca7b88133229 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.559165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ad69d-0e05-4208-8017-ca7b88133229 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.559529 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ad69d-0e05-4208-8017-ca7b88133229 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.560310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ad69d-0e05-4208-8017-ca7b88133229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:03.560707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ad69d-0e05-4208-8017-ca7b88133229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.562341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ad69d-0e05-4208-8017-ca7b88133229 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.583553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 272/538] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:03.585248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27a0e1-cbbc-4fb7-a44f-ab1b873149e4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.585604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27a0e1-cbbc-4fb7-a44f-ab1b873149e4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.585973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27a0e1-cbbc-4fb7-a44f-ab1b873149e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.609552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6d27a0e1-cbbc-4fb7-a44f-ab1b873149e4 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:03.610889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6d27a0e1-cbbc-4fb7-a44f-ab1b873149e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:03.656295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4bf77-2b94-44ca-b2cd-d0dc82f094a3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:03.701730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.702741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:03.703333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.703765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.704173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.722275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.724137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.736641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 273/539] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:03.736641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27a0e1-cbbc-4fb7-a44f-ab1b873149e4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:03.762462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:03.763573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:03.764653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:03.768489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 267/540] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:03.771784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:03.878710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:03.899579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d781d73-0cbc-40e4-b178-052ccdd1db53 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:03.927294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27a0e1-cbbc-4fb7-a44f-ab1b873149e4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:03.958416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.959279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.959568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.959839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.960599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:03.960914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.962559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.968186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:03.969185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:03.969658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:03.970118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:03.971051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=a4b10367002d4f64a82b3e9eca00ca06) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:03.971553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:03.973253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:03.979921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 274/541] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/role_assignments?user.id=a4b10367002d4f64a82b3e9eca00ca06&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:03.986456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 268/542] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:03.992834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:04.056452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:04.122012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:04.125210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62e0b5-1457-4a43-aaef-727f622d2db6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:04.196466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91cc24-7ce7-4558-93d6-b151c83e11bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:04.196996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91cc24-7ce7-4558-93d6-b151c83e11bb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:04.197272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91cc24-7ce7-4558-93d6-b151c83e11bb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:04.197540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91cc24-7ce7-4558-93d6-b151c83e11bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:04.198295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91cc24-7ce7-4558-93d6-b151c83e11bb None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=2c282d6f237542e3b873100748759794) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:04.198601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91cc24-7ce7-4558-93d6-b151c83e11bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:04.200089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91cc24-7ce7-4558-93d6-b151c83e11bb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:04.206962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 275/543] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:03 2022] GET /identity/v3/role_assignments?user.id=2c282d6f237542e3b873100748759794&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:04.305949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38e57-8491-4d6b-96c6-6ddd7ad2eb49 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:04.306294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38e57-8491-4d6b-96c6-6ddd7ad2eb49 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:04.306500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38e57-8491-4d6b-96c6-6ddd7ad2eb49 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:04.308177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 276/544] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:48:04.316228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b3eae-6af4-4533-afcf-a763c65e11c2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:04.316566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b3eae-6af4-4533-afcf-a763c65e11c2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:04.316807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b3eae-6af4-4533-afcf-a763c65e11c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:04.317192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812adea-b59d-4a2e-ad87-863ec6156c56 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:04.348366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-056b3eae-6af4-4533-afcf-a763c65e11c2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:04.350631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-056b3eae-6af4-4533-afcf-a763c65e11c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:04.369880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:04.370700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:04.371164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:04.371538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:04.408071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:04.424276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:04.425275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:04.426077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:04.432183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 269/545] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:04.448158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:04.456003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:04.506321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 277/546] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:04.588235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f94fa-088e-421f-b1b9-fa159f6b1b38 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:04.625343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:04.625973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:04.626239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:04.626504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:04.627270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:04.627573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:04.629055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:04.639157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 270/547] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:04.644459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b3eae-6af4-4533-afcf-a763c65e11c2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:04.722901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:04.806553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b3eae-6af4-4533-afcf-a763c65e11c2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:04.855862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:04.857817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:04.858474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:04.858860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:04.859134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:04.876680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:04.878283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:04.904713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:04.905754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:04.906588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:04.906588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:04.906588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:04.906588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 278/548] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:04.912205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:04.934932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:04.973514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565d39a-0328-4ef1-8fd7-b256ec3f96a7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.009558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.010203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.010518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.010928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.023136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:05.028690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:05.031028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:05.032986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 271/549] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:05.040473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:05.094693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:05.107804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd911a17-0f70-4576-b15c-942c805ee8e7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.167781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7bfe7-e2b1-431e-b319-65d070608a3a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.193516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.194169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.194444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.194707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.195387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:05.195687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:05.197207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:05.206547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:05.225296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.226358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.226855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.227442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.228499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:05.229109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:05.230982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:05.231248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 279/550] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:05.237047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:05.244028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 272/551] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:05.251551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:05.401353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce986e-32c6-4d60-bafc-b48f6f757251 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.443894 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.445240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.445843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.446398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.459968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:05.461984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:05.479299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:05.480241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:05.481167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:05.481167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:05.481167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:05.483490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 273/552] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:05.490074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:05.560226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8b3d5-7945-47ce-808e-1dd7fbf3ac41 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.574110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:05.615442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.616122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.616405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.616993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.635038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:05.637246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:05.652357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c859414-cd7a-45d7-adec-137bbe893738 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:05.661759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:05.663245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:05.664181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 280/553] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:48:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:05.669084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:05.698068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.699123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.699664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.700123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.701139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:05.701645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:05.703311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:05.705050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:05.720373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 274/554] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:05.726453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:05.824032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ed572-9098-4a15-8ade-24e8c5a37dd8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.842652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:05.847688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca517cf-bd86-4d9e-8c26-2a75036101a3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:05.887857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.888945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.889416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.889877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.898677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:05.900565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:05.913798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:05.914897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:05.916271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:05.919026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 275/555] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:05.925038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:05.945412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:05.946542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:05.947155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:05.947501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:05.948925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:05.949424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:05.952191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.016947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 281/556] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:48:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:06.023683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:06.041460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70efe3d-ec39-4e31-870f-26290166d43b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:06.082997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:06.084088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.084622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.085106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.086063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:06.086582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:06.088277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.112588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 276/557] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 13 17:48:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:06.114621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:06.136024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:06.180898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:06.225961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577096c-2411-4ea7-8a2c-f644071da591 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:06.234747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8c7de-4003-4a46-b5bd-2856001448e8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:06.274950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:06.275938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.276456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.276917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.290502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:06.291171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.291455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.291731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.319736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:06.321264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:06.353563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:06.354594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:06.355523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:06.355523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:06.355523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:06.355523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 282/558] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:06.358511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:06.360293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.363413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:06.381691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:06.510562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 277/559] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:06.517003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:06.534773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fa938-c3d0-4193-82bf-b1ba69b138d7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:06.583059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:06.602324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:06.602972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.603259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.603528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.604196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:06.604534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:06.605946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.619752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 283/560] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:06.628366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:06.639041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dc1ec-1430-437e-94a5-32544eb7c4f1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:06.660248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:06.683637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceb570-0f77-4732-ab15-79d4e21fdde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:06.684678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceb570-0f77-4732-ab15-79d4e21fdde1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.685150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceb570-0f77-4732-ab15-79d4e21fdde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.685889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceb570-0f77-4732-ab15-79d4e21fdde1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.700935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceb570-0f77-4732-ab15-79d4e21fdde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:06.703127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceb570-0f77-4732-ab15-79d4e21fdde1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-aaceb570-0f77-4732-ab15-79d4e21fdde1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:06.723957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:06.725282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:06.726351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:06.726351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:06.726351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:06.728315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 278/561] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:06.734248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcde73-b6c8-425d-9426-2419ef1be0d4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.736638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcde73-b6c8-425d-9426-2419ef1be0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.737222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcde73-b6c8-425d-9426-2419ef1be0d4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.739215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 279/562] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:06.740364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcde73-b6c8-425d-9426-2419ef1be0d4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:06.777752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3414b2f-582f-4bff-940a-80cf2baf223b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:06.822001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460484c-8268-4078-b162-ff4d0a364e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:06.822885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460484c-8268-4078-b162-ff4d0a364e2a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.823000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460484c-8268-4078-b162-ff4d0a364e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.823291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460484c-8268-4078-b162-ff4d0a364e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.832178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460484c-8268-4078-b162-ff4d0a364e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:06.834006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460484c-8268-4078-b162-ff4d0a364e2a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e460484c-8268-4078-b162-ff4d0a364e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:06.848677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:06.849716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:06.850585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 284/563] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:06.852181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9aab3a-7dad-40ea-bbc1-c0ab255d893c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.852595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9aab3a-7dad-40ea-bbc1-c0ab255d893c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.852963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9aab3a-7dad-40ea-bbc1-c0ab255d893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.873940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2b9aab3a-7dad-40ea-bbc1-c0ab255d893c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:06.875414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2b9aab3a-7dad-40ea-bbc1-c0ab255d893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:06.902278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcde73-b6c8-425d-9426-2419ef1be0d4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:06.930427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:06.948105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628a78a-b1eb-4f40-b3cc-de65af48f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:06.948982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628a78a-b1eb-4f40-b3cc-de65af48f796 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.949341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628a78a-b1eb-4f40-b3cc-de65af48f796 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.949654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628a78a-b1eb-4f40-b3cc-de65af48f796 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.950430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628a78a-b1eb-4f40-b3cc-de65af48f796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:06.950830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628a78a-b1eb-4f40-b3cc-de65af48f796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:06.952751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628a78a-b1eb-4f40-b3cc-de65af48f796 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:06.972491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 285/564] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:06.972491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9aab3a-7dad-40ea-bbc1-c0ab255d893c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:06.972839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 280/565] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:06.972839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a2500-a022-4a75-ba5f-4ece9764e5bd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.972839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a2500-a022-4a75-ba5f-4ece9764e5bd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.972839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a2500-a022-4a75-ba5f-4ece9764e5bd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:06.972839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 281/566] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:06.977369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc102596-dd9b-4fa7-aadf-a95ae9a789fb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:06.977650 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc102596-dd9b-4fa7-aadf-a95ae9a789fb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:06.977926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc102596-dd9b-4fa7-aadf-a95ae9a789fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.003629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cc102596-dd9b-4fa7-aadf-a95ae9a789fb None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:07.005288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cc102596-dd9b-4fa7-aadf-a95ae9a789fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:07.038938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:07.089167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9aab3a-7dad-40ea-bbc1-c0ab255d893c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:07.096952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 282/567] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:07.098135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc102596-dd9b-4fa7-aadf-a95ae9a789fb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:07.138304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:07.138965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.139247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.139516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.140244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:07.140588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:07.142092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:07.156103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 286/568] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:07.165528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:07.247044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:07.307238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc102596-dd9b-4fa7-aadf-a95ae9a789fb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:07.347128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ed26b-bc37-45bf-8ba2-b581faba5314 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:07.354980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d657cc0-f9b6-4d4b-98aa-3383564c6daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:07.355625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d657cc0-f9b6-4d4b-98aa-3383564c6daf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.355903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d657cc0-f9b6-4d4b-98aa-3383564c6daf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.356173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d657cc0-f9b6-4d4b-98aa-3383564c6daf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.363715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d657cc0-f9b6-4d4b-98aa-3383564c6daf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:07.365306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d657cc0-f9b6-4d4b-98aa-3383564c6daf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6d657cc0-f9b6-4d4b-98aa-3383564c6daf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:07.377471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:07.378426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:07.379298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 283/569] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:07.379808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6ecdb-01bc-422a-bb81-a86f5bbc9c2f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.380413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6ecdb-01bc-422a-bb81-a86f5bbc9c2f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.380413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6ecdb-01bc-422a-bb81-a86f5bbc9c2f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.381864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 284/570] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:07.386129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6ecdb-01bc-422a-bb81-a86f5bbc9c2f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:07.408750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6c9fd-0831-4bf1-a984-28e5fd9936b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:07.409429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6c9fd-0831-4bf1-a984-28e5fd9936b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.409706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6c9fd-0831-4bf1-a984-28e5fd9936b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.409972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6c9fd-0831-4bf1-a984-28e5fd9936b2 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/90f40fe8f20f4a5bbd201debda6a97a5/roles/cb2c16a6113142179869b36ff210c590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.413714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:07.426486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6c9fd-0831-4bf1-a984-28e5fd9936b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=90f40fe8f20f4a5bbd201debda6a97a5, role_id=cb2c16a6113142179869b36ff210c590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:07.428008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6c9fd-0831-4bf1-a984-28e5fd9936b2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:07.446734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 287/571] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:07 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/90f40fe8f20f4a5bbd201debda6a97a5/roles/cb2c16a6113142179869b36ff210c590 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:07.449374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8c6a1-1089-4191-be59-8d088cbe225b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.449847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8c6a1-1089-4191-be59-8d088cbe225b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.450260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8c6a1-1089-4191-be59-8d088cbe225b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.480027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-92e8c6a1-1089-4191-be59-8d088cbe225b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:07.481965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-92e8c6a1-1089-4191-be59-8d088cbe225b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:07.516219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6ecdb-01bc-422a-bb81-a86f5bbc9c2f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:07.558774 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:07.559753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.559753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.560022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.574108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:07.579352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:07.586449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 285/572] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:07.587277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:07.610786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 288/573] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:07.612303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8c6a1-1089-4191-be59-8d088cbe225b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:07.648595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:07.718584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:07.760994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65788a11-8d35-496b-922d-69e325d931a1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:07.789933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8c6a1-1089-4191-be59-8d088cbe225b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:07.823188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:07.823827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.824111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.824374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.825142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:07.825447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:07.827567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:07.844495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 286/574] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:07.844495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:07.859477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:07.860888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:07.861389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:07.861947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:07.883034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:07.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:07.894497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:07.895312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:07.896733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 289/575] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:07.897461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:07.964738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26efdd-78f8-48f2-9e4d-b3e1f33ac18d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.014311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.015104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.015510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.015978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.016891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:08.017374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:08.019310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:08.030926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 287/576] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:08.036489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:08.037832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:08.038798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:08.074202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44a8f-2343-4e7b-9303-637297168a3d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.115697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.116199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.116524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.116771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.117523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1ec763f1afac469196a5e897587813b8) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:08.117824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:08.119281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:08.125672 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 290/577] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:07 2022] GET /identity/v3/role_assignments?user.id=1ec763f1afac469196a5e897587813b8&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:08.126538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:08.253310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc2289-b66c-4cb8-b62c-fd89813a8737 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.322524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.323482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.323920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.324467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.339420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bac5f-128a-477d-82c1-bd112b3a9b6a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.344322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:08.356786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:08.358561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:08.360193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:08.365004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 288/578] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:08.367197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:08.412062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.413128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.413600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.414072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.415258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:08.415759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:08.418588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:08.440709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 291/579] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:08.442763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:08.448785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:08.460579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:08.515977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993ac2a-a242-4f82-a229-42fb105e0d02 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.560644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.561935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.562356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.563266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.576894 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:08.578959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:08.597462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:08.598607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:08.599714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:08.599714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:08.599714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:08.601810 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 289/580] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:08.602848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:08.630208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63723c3-64b1-4cce-9537-f18890dfadd9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.694815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.695779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.696398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.696971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.698195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:08.698716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:08.701354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:08.717905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 292/581] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:08.719971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:08.756478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc01554-b3c3-45f9-bc22-c920c515f370 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.777008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:08.808517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.809946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.810428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.810855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.826373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:08.828516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:08.837839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:08.853283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:08.854518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:08.855573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:08.855573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:08.855573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:08.859573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 290/582] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:08.861360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:08.921166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27e6c0-2fd4-46d4-a44a-fa858fa0ff3f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:08.991439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:08.992801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:08.993884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:08.994137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:08.996072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:08.996373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:08.999488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:09.017050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc0f92-36eb-4699-bc2b-6d9427b31e41 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:09.032736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 293/583] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:09.034602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:09.064006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:09.064968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:09.065418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:09.065896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:09.082544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:09.084825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:09.101945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:09.102965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:09.103839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:09.103839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:09.103839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:09.106253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 291/584] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:09.107426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:09.144921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:09.212227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd3959-088a-4429-9057-ff5ddb99edfd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:09.261131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:09.280296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:09.281656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:09.282051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:09.282609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:09.283668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:09.284395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:09.286867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:09.298146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8cbc3d-3922-409a-b9f5-04135fe30fc4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:09.308708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 294/585] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:09.311315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:09.372393 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:09.374592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:09.375075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:09.375535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:09.393766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:09.397357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:09.423918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:09.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:48:09.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:09.427103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 292/586] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:09.428482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:09.499949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884433d9-8fab-4ac8-9024-7b973d0618b4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:09.574248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:09.575364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:09.575819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:09.577083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:09.578228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:09.578730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:09.581879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:09.583808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:09.609796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 295/587] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:09.611246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:09.635659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a58c4-ba15-48f4-ad07-da89e2bbd345 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:09.710382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:09.718814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:09.720114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:09.720611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:09.721058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:09.744627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:09.747539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:09.775071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:09.776999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:09.778842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 293/588] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:09.781411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:09.835906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b5938-5ce6-452e-a781-113427fefb14 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:09.905926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:09.906973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:09.907411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:09.907822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:09.909593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:09.910110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:09.913873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:09.953705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 296/589] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 13 17:48:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:09.958615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:09.989626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0aa6e8-6741-43fb-95ae-134f66659cc3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:10.049165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:10.050279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.050921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.051389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.067490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:10.069358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:10.074053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:10.089052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:10.090267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 17:48:10.091207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:10.100593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 294/590] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:10.100593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:10.150205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3840a17-9fd8-419a-811a-ae338a4cb85a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:10.152655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:10.196206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ffb48-42a0-47f5-929e-8e183283d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:10.196913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ffb48-42a0-47f5-929e-8e183283d59e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.197173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ffb48-42a0-47f5-929e-8e183283d59e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.197621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ffb48-42a0-47f5-929e-8e183283d59e None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.199283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ffb48-42a0-47f5-929e-8e183283d59e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:10.199617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ffb48-42a0-47f5-929e-8e183283d59e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:10.201151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ffb48-42a0-47f5-929e-8e183283d59e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:10.234475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 297/591] 10.222.0.43 () {56 vars in 1139 bytes} [Tue Sep 13 17:48:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:10.235021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a957f-7e5b-4bff-9e0e-1f02332339d7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.235412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a957f-7e5b-4bff-9e0e-1f02332339d7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.235885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a957f-7e5b-4bff-9e0e-1f02332339d7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.238673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 298/592] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:10.240030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a957f-7e5b-4bff-9e0e-1f02332339d7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:10.250972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8a053-6240-43bc-9de7-292b67302f43 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:10.295142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:10.296116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.296586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.297018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.314996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:10.317026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:10.338598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:10.339606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:10.340516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:10.340516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:10.340516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:10.340516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 295/593] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:10.341361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:10.402838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a957f-7e5b-4bff-9e0e-1f02332339d7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:10.422849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:10.440973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a61b5-6dc5-4ae7-b974-1fd061b305b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:10.441710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a61b5-6dc5-4ae7-b974-1fd061b305b0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.441981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a61b5-6dc5-4ae7-b974-1fd061b305b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.442273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a61b5-6dc5-4ae7-b974-1fd061b305b0 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.443166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a61b5-6dc5-4ae7-b974-1fd061b305b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:10.443615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a61b5-6dc5-4ae7-b974-1fd061b305b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:10.445163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a61b5-6dc5-4ae7-b974-1fd061b305b0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:10.464658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 299/594] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 13 17:48:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:10.465756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747112c-ab47-438a-b327-83514ecf5d2b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.466039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747112c-ab47-438a-b327-83514ecf5d2b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.466306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747112c-ab47-438a-b327-83514ecf5d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.484935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f747112c-ab47-438a-b327-83514ecf5d2b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:10.486140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f747112c-ab47-438a-b327-83514ecf5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:10.504483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f762a-e76a-4b41-84bb-ac1f84298ef5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:10.505463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:10.541392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:10.542032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.542307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.542573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.554737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:10.556238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:10.572739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:10.573722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:10.574587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:10.574587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:10.574587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:10.578495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 296/595] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:10.579314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:10.597523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 300/596] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:10.598297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747112c-ab47-438a-b327-83514ecf5d2b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:10.745562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747112c-ab47-438a-b327-83514ecf5d2b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:10.794094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:10.794854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.795192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.795528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.807642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:10.809654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:10.819866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:10.826636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25c194-cde9-4eab-bada-3945a4de8a36 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:10.829121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:10.830890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:10.833196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:10.833196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:10.833196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:10.835072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 301/597] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:10.835682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:10.836068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:10.921337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:10.924025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:10.924814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:10.925540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:10.927005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:10.927849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:10.931928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:10.947216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 297/598] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:10.952811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:10.960647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c363-9cd5-4997-aff0-ef08f6f96a94 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.014420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.015753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.016243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.016725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.017964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:11.018470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.021111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:11.040528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 302/599] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:11.043384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.115271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbf478-c927-48d2-b630-8e3e0f83de8d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.147811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:11.173370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.174162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.174562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.174958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.175784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:11.176213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.178261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:11.199631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 298/600] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:11.199631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.231808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cd9e9-0ffa-4f8a-8b2d-6665c60e4bec None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.236452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:11.289109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.290204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.290667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.291153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.308122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:11.318146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:11.319232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:11.320096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:11.320790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 303/601] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:11.321738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.363781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f788d-0507-4b19-91ed-008741c3bc5f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.424212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.425213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.425687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.426137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.439568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.442295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:11.471303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:11.473727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 299/602] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:11.475522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.515272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd131e8b-3c66-4390-9f78-8a32a6983f24 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.557561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:11.584053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.585219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.585676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.586126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.598307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.600120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:11.614744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:11.616012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:11.617107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:11.617535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:11.617762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 304/603] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:11.622819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.644934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fb8c2-ac07-4e38-9f14-495e4680f39d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.702452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.703253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.703735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.704214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.716154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.719193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:11.736963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:11.738867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 300/604] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:11.740414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.741347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c95fb-1a47-451b-b146-1eae81bc2490 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.796669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9c6b9-7a31-41e8-9df3-afc3cdc2d8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.797280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9c6b9-7a31-41e8-9df3-afc3cdc2d8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.797558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9c6b9-7a31-41e8-9df3-afc3cdc2d8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.798119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9c6b9-7a31-41e8-9df3-afc3cdc2d8f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.798522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9c6b9-7a31-41e8-9df3-afc3cdc2d8f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:11.798824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9c6b9-7a31-41e8-9df3-afc3cdc2d8f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.800253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9c6b9-7a31-41e8-9df3-afc3cdc2d8f9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:11.809946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 305/605] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:11.811280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b576fba-a666-4ead-9e82-20bf7f7ed04b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.811570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b576fba-a666-4ead-9e82-20bf7f7ed04b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.811844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b576fba-a666-4ead-9e82-20bf7f7ed04b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.813351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 306/606] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:11.813881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b576fba-a666-4ead-9e82-20bf7f7ed04b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.876051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:11.886172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf1488-56b4-4e9e-9250-5e97d1a99d14 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.923224 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.923666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.923955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.924244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.924969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:11.925339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.926763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:11.939571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 301/607] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:11.940449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:11.943250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:11.951476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b576fba-a666-4ead-9e82-20bf7f7ed04b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:11.988303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:11.989702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:11.989976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:11.990275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:11.990995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:11.991316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:11.992970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:12.021857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 307/608] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:12.022745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:12.069216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbcf02-cfdc-4f07-b759-665b6899f606 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:12.104506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedf101-0131-4fa1-9af0-f7f19c299e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:12.105131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedf101-0131-4fa1-9af0-f7f19c299e85 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:12.105415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedf101-0131-4fa1-9af0-f7f19c299e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:12.105682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedf101-0131-4fa1-9af0-f7f19c299e85 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:12.106397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedf101-0131-4fa1-9af0-f7f19c299e85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:12.106705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedf101-0131-4fa1-9af0-f7f19c299e85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:12.108159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedf101-0131-4fa1-9af0-f7f19c299e85 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:12.120953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 302/609] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:12.122409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d4f08-fde1-48b6-b529-ea2e58246ca4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:12.122700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d4f08-fde1-48b6-b529-ea2e58246ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:12.122980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d4f08-fde1-48b6-b529-ea2e58246ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:12.134563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300822a-74ea-420e-b924-e4e82f948d8d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:12.137749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:12.143721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c26d4f08-fde1-48b6-b529-ea2e58246ca4 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:12.145185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c26d4f08-fde1-48b6-b529-ea2e58246ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:12.182406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:12.183212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:12.183497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:12.183768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:12.194831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:12.196365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:12.212643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:12.214040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:12.214954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:12.214954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:12.214954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:12.215466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 308/610] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:12.216676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:12.217156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:12.229337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 303/611] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:12.230189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d4f08-fde1-48b6-b529-ea2e58246ca4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:12.362691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb007c18-5663-4da8-98d4-34eca64f961d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:12.379681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d4f08-fde1-48b6-b529-ea2e58246ca4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:12.407091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:12.407951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:12.408359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:12.408848 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/6c9c5205f9344e8c8fcc85c80d6f504b/roles/cb2c16a6113142179869b36ff210c590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:12.423064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:12.424011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:12.424285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:12.424591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/a4b10367002d4f64a82b3e9eca00ca06/roles/cb2c16a6113142179869b36ff210c590` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:12.429299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=6c9c5205f9344e8c8fcc85c80d6f504b, role_id=cb2c16a6113142179869b36ff210c590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:12.431577 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:12.446398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=a4b10367002d4f64a82b3e9eca00ca06, role_id=cb2c16a6113142179869b36ff210c590)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:12.456184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:12.456410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 309/612] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:12 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/6c9c5205f9344e8c8fcc85c80d6f504b/roles/cb2c16a6113142179869b36ff210c590 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:12.456410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:12.463961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:12.497918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 304/613] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:12 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/a4b10367002d4f64a82b3e9eca00ca06/roles/cb2c16a6113142179869b36ff210c590 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:12.500140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:12.587077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:12.645438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980300e6-6a9a-4c63-a4b2-a939f13689d3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:12.705526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bddc74-e9ec-4a1b-8a8f-3381e5de462f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:12.707409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:12.708485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:12.708485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:12.708485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/2c282d6f237542e3b873100748759794/roles/cb2c16a6113142179869b36ff210c590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:12.745161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=2c282d6f237542e3b873100748759794, role_id=cb2c16a6113142179869b36ff210c590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:12.748327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:12.773978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 310/614] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:12 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/2c282d6f237542e3b873100748759794/roles/cb2c16a6113142179869b36ff210c590 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:12.776129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:12.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:12.790492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:12.790967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:12.791422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:12.793473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:12.794096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:12.796250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:12.821546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 305/615] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:12.822994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:12.884986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:12.998110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a8e7-9bb0-4c3e-9041-8b5c6d79468f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:13.048355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a1648-f168-4e7e-834e-dc57db4a4b9b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:13.049264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:13.049929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:13.050204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:13.050472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:13.053299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:13.070432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:13.075426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:13.077629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:13.079576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 311/616] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:13.083156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:13.122276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:13.123096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:13.123594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:13.124003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:13.141125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:13.142868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:13.158625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:13.160257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:13.161531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 306/617] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:48:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:13.169979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:13.251163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb501-3714-40a6-9cfc-48810228c8c1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:13.281351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:13.322085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:13.323661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:13.324470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:13.325398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:13.327349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:13.328137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:13.330814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:13.342241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 312/618] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:13.348554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:13.456790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6591b0b-e256-426d-8851-20e50d79acd7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:13.502003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d580afd-a1fe-4f89-ad39-2b6dd752697d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:13.533658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:13.534459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:13.534829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:13.535191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:13.536051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:13.536532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:13.538928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:13.571824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 307/619] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:48:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:13.575855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:13.576727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:13.577098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:13.577456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:13.580286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:13.601871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:13.604233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:13.624798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:13.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:13.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:13.635750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:13.635750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:13.635750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:13.641142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 313/620] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:13.644196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:13.720414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:13.803491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792939f7-07d8-4239-923f-33fb80103565 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:13.852507 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:13.853302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:13.853587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:13.853885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:13.975296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e2bed-fca6-44b3-9073-b51d9d6b5184 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:14.048753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:14.051439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:14.092551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:14.093102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:14.093366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:14.093630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:14.094293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:14.094592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:14.095985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:14.112836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:14.132003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 314/621] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:14.140080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:14.279586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 308/622] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 17:48:14.284013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:14.377671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:14.414781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af5f9b-4654-4d8c-99f7-d155c9fe6791 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:14.530862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:14.531937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:14.532220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:14.532594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:14.541180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:14.542678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:14.556408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:14.557491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:14.558490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 315/623] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:14.574734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:14.601197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab80bd-6a34-4b6d-bdd1-64eb9ebc45fa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:14.660175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:14.703469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:14.704595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:14.705113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:14.705618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:14.739912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:14.742331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:14.768159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87fe5b-abaf-4612-a403-994cd69a1abf None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:14.826407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:14.827436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:14.828357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:14.828357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:14.828357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:14.829900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 309/624] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 546 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:14.835631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:14.836974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:14.837547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:14.838160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:14.839154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:14.839761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:14.841536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:14.842115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:14.867933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 316/625] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 13 17:48:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:14.875267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:14.896919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:15.077982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8b58e-2b06-4285-9e90-a6e828e5f01b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:15.168723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f7365-866c-494b-ba52-1a34b4958ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:15.170654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f7365-866c-494b-ba52-1a34b4958ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.171148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f7365-866c-494b-ba52-1a34b4958ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.171624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f7365-866c-494b-ba52-1a34b4958ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.195587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f7365-866c-494b-ba52-1a34b4958ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:15.199701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f7365-866c-494b-ba52-1a34b4958ab8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-672f7365-866c-494b-ba52-1a34b4958ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:15.222908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:15.223940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:15.225958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:15.225958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:15.225958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:15.232836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 317/626] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:15.234791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f92a5-da4e-4900-85f0-30b749b934dc None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.235402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f92a5-da4e-4900-85f0-30b749b934dc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.235912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f92a5-da4e-4900-85f0-30b749b934dc None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.238477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 318/627] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:15.239705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f92a5-da4e-4900-85f0-30b749b934dc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:15.287918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a939e-91e9-467a-8028-688342a2dc25 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:15.336327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:15.388710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbc0c7-b5d7-4c79-9b47-d879e5bd0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:15.389353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc0c7-b5d7-4c79-9b47-d879e5bd0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.389634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc0c7-b5d7-4c79-9b47-d879e5bd0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.389940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc0c7-b5d7-4c79-9b47-d879e5bd0cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.390716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbc0c7-b5d7-4c79-9b47-d879e5bd0cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:15.391025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbc0c7-b5d7-4c79-9b47-d879e5bd0cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:15.392485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbc0c7-b5d7-4c79-9b47-d879e5bd0cb9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:15.405509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 310/628] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 564 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:15.406801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8f6e5-8e32-4925-bc77-274b77525bc0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.407085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8f6e5-8e32-4925-bc77-274b77525bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.407360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8f6e5-8e32-4925-bc77-274b77525bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.427211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e1b8f6e5-8e32-4925-bc77-274b77525bc0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:15.428518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e1b8f6e5-8e32-4925-bc77-274b77525bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:15.443565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f92a5-da4e-4900-85f0-30b749b934dc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:15.466925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:15.501226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bc807-91f9-4c11-ae61-429e8919cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:15.501868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc807-91f9-4c11-ae61-429e8919cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.502853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc807-91f9-4c11-ae61-429e8919cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.502853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bc807-91f9-4c11-ae61-429e8919cbd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.503401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc807-91f9-4c11-ae61-429e8919cbd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:15.504022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc807-91f9-4c11-ae61-429e8919cbd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:15.505961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bc807-91f9-4c11-ae61-429e8919cbd9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:15.524630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 311/629] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:15.524630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bebb96-424a-4120-801d-4602bc85b21a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.524630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bebb96-424a-4120-801d-4602bc85b21a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.524630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bebb96-424a-4120-801d-4602bc85b21a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.524630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 312/630] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:15.524630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bebb96-424a-4120-801d-4602bc85b21a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:15.540640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 319/631] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:15.542191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98bdc-8a67-48d4-8bd6-eea1e048254e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.542538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98bdc-8a67-48d4-8bd6-eea1e048254e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.542824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98bdc-8a67-48d4-8bd6-eea1e048254e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.562527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a1c98bdc-8a67-48d4-8bd6-eea1e048254e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:15.563771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a1c98bdc-8a67-48d4-8bd6-eea1e048254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:15.707964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bebb96-424a-4120-801d-4602bc85b21a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:15.709306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 320/632] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:15.711527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c98bdc-8a67-48d4-8bd6-eea1e048254e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:15.798670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:15.799391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.799668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.799929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.813228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:15.815935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:15.819532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:15.839539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:15.841423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:15.843174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 313/633] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:15.855908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:15.909214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:15.940753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c98bdc-8a67-48d4-8bd6-eea1e048254e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:15.987850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656c36c-40c5-4423-8f7e-8554defa1a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:15.988655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656c36c-40c5-4423-8f7e-8554defa1a52 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:15.989077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656c36c-40c5-4423-8f7e-8554defa1a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:15.989502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656c36c-40c5-4423-8f7e-8554defa1a52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:15.997978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656c36c-40c5-4423-8f7e-8554defa1a52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:15.999599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656c36c-40c5-4423-8f7e-8554defa1a52 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c656c36c-40c5-4423-8f7e-8554defa1a52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:16.014784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:16.016923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:16.018790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:16.022825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 321/634] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:16.024382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeabfb-e2db-4f2d-844e-71bac049ce5a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.024818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeabfb-e2db-4f2d-844e-71bac049ce5a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.025189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeabfb-e2db-4f2d-844e-71bac049ce5a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.026973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 322/635] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:16.027856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeabfb-e2db-4f2d-844e-71bac049ce5a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.091932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcc2ae-edc3-44c8-a4f6-218a8c8edd3e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.133008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e189c5-6f2a-4f9f-ab84-149001b5f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.133617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e189c5-6f2a-4f9f-ab84-149001b5f668 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.133913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e189c5-6f2a-4f9f-ab84-149001b5f668 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.134165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e189c5-6f2a-4f9f-ab84-149001b5f668 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.134866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e189c5-6f2a-4f9f-ab84-149001b5f668 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:16.135165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e189c5-6f2a-4f9f-ab84-149001b5f668 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:16.137026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e189c5-6f2a-4f9f-ab84-149001b5f668 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:16.157505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 314/636] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:16.157505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b951cd-99f2-4982-be30-c2a5d1c83cc1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.157505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b951cd-99f2-4982-be30-c2a5d1c83cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.157505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b951cd-99f2-4982-be30-c2a5d1c83cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.158840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeabfb-e2db-4f2d-844e-71bac049ce5a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.171148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:16.174799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-13b951cd-99f2-4982-be30-c2a5d1c83cc1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:16.176016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-13b951cd-99f2-4982-be30-c2a5d1c83cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:16.202936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.203706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.203979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.204242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.204984 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:16.205286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:16.206735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:16.219682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 323/637] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:16.225874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.257437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 315/638] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:16.258388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b951cd-99f2-4982-be30-c2a5d1c83cc1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.275426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:16.357344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24256429-ec97-4340-983a-9c16c2569035 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.401259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.401920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.402198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.402482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.410207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b951cd-99f2-4982-be30-c2a5d1c83cc1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.421408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:16.432551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:16.434506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:16.436270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 324/639] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:16.437606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.467114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.467777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.468091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.468365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.481600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:16.486619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:16.488963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 316/640] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:16.490772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.504273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:16.578806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:16.605826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec68fe-d37a-4310-9821-3913f472b022 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.616157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35575f6f-a5a8-40c6-b700-5407f557f6ca None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.652012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.653072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.653539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.654024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.655437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=90f40fe8f20f4a5bbd201debda6a97a5) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:16.655878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:16.657378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:16.661897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.662591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.662861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.663126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/1ec763f1afac469196a5e897587813b8/roles/cb2c16a6113142179869b36ff210c590` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.663706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 325/641] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:16 2022] GET /identity/v3/role_assignments?user.id=90f40fe8f20f4a5bbd201debda6a97a5&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:16.664562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.683960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=1ec763f1afac469196a5e897587813b8, role_id=cb2c16a6113142179869b36ff210c590)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:16.685561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:16.701161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 317/642] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:16 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/1ec763f1afac469196a5e897587813b8/roles/cb2c16a6113142179869b36ff210c590 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:16.702046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.873669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd39fa-e168-4efa-b69b-63936bd64ae5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.888881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:16.893863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d13c-9e1e-41ab-8fa7-e54671ef0af4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:16.929203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.929859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.930134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.930401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.931115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:16.931415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:16.933211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:16.944285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 326/643] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:16.944704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:16.945086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:16.976708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:16.978814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:16.979556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:16.980496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:16.982001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:16.982957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:16.986283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:17.003611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 318/644] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:17.005203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:17.075096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857df1f-4092-4c89-82c7-1d676be96d96 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:17.119350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:17.120556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:17.121016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:17.121309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:17.137518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:17.143924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:17.145052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:17.145917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:17.146626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 327/645] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:17.147196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:17.166537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:17.184934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdee64b-05a0-4ebe-acb9-a56bfa2166df None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:17.227446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:17.228550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:17.229220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:17.229737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:17.245422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:17.248329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:17.269000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce1d9b-0dbb-46b5-aee9-36e0a873a38a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:17.270574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:17.272121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:17.273261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:17.273261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:17.273261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:17.276194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 319/646] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:17.277729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:17.310113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:17.310766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:17.311037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:17.311299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:17.324405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:17.326254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:17.341311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:17.344589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:17.345682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:17.346694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:17.346694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:17.346694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:17.353630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 328/647] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:17.355034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:17.449162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713af900-0cf3-48f2-bc1a-128fbd2b334d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:17.477856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:17.497037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:17.497929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:17.498374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:17.498806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:17.499660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:17.500113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:17.501852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:17.516552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 320/648] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:17.516552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:17.560593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab78a7-6155-4a31-8225-68fd10ba96e6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:17.636485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106701-8396-49ce-bb38-697e228008cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:17.638243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106701-8396-49ce-bb38-697e228008cb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:17.638932 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106701-8396-49ce-bb38-697e228008cb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:17.640646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48106701-8396-49ce-bb38-697e228008cb None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:17.642115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106701-8396-49ce-bb38-697e228008cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:17.643718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106701-8396-49ce-bb38-697e228008cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:17.648159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48106701-8396-49ce-bb38-697e228008cb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:17.671268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:17.678303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f96fba-04b1-441c-8b1c-286dddc072b0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:17.682713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 329/649] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:17.686774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106701-8396-49ce-bb38-697e228008cb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:17.751016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:17.752629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:17.753405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:17.754134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:17.755668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:17.756666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:17.759184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:17.800046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 321/650] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:17.800046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:17.909231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48106701-8396-49ce-bb38-697e228008cb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:17.960962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:17.981608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:17.982988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:17.983457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:17.983931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:18.008256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:18.012268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:18.022913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d3322-e500-4fd5-ba95-da00f9b812f2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:18.046318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:18.048157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:18.049925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:18.049925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:18.049925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:18.052739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 330/651] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:18.055134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:18.102027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:18.103844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:18.104629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:18.105325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:18.120414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:18.123363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:18.146097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:18.148021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:48:18.150348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:18.153851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 322/652] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:18.156089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:18.193324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:18.280582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec689270-6ef2-41ec-a698-cb0a88d059d3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:18.348609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:18.350128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:18.350852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:18.351584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:18.367827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:18.368397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67112779-1eda-452b-8cb8-d93f0f7b145e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:18.371076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:18.397656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:18.399366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 17:48:18.401141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:18.404538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 331/653] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:18.406479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:18.448152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:18.449225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:18.449708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:18.450151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:18.451331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:18.451841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:18.456014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:18.488548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:18.488804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 323/654] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:18.491109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:18.617589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc26e5-51f7-4c03-ba74-692c46981e68 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:18.667629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:18.693481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:18.695043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:18.695816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:18.696713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:18.698855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:18.699738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:18.703057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:18.717078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1735ad-0c81-4922-b911-0ac710d13276 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:18.746515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 332/655] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 13 17:48:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:18.748612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:18.793370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:18.794417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:18.794905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:18.795374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:18.797170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:18.797701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:18.800797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:18.843441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 324/656] 10.222.0.43 () {56 vars in 1139 bytes} [Tue Sep 13 17:48:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:18.845604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:18.957316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:18.973472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d99613-637f-4693-ac1c-3f7680339891 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:19.046247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:19.047344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.047832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.048280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.055464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0090d5-7833-4bb5-82e0-168caa7426e2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:19.065094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:19.067785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:19.094515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:19.096357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 333/657] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:19.098303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:19.142279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca09b2-5c45-41f1-856d-fb8bdfbdf805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:19.143688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca09b2-5c45-41f1-856d-fb8bdfbdf805 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.144726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca09b2-5c45-41f1-856d-fb8bdfbdf805 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.146223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca09b2-5c45-41f1-856d-fb8bdfbdf805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.154870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:19.232816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca09b2-5c45-41f1-856d-fb8bdfbdf805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:19.234227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca09b2-5c45-41f1-856d-fb8bdfbdf805 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:19.273341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e858-e01c-47ba-abf3-9fc0a6360d3d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:19.311690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41274-7999-49a5-846b-edeac195fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:19.312383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41274-7999-49a5-846b-edeac195fd01 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.312630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41274-7999-49a5-846b-edeac195fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.312904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41274-7999-49a5-846b-edeac195fd01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.341451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:19.383773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 325/658] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:19.385232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa10c4c-c2c9-4eb3-a41d-c907852c12e2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.385528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa10c4c-c2c9-4eb3-a41d-c907852c12e2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.385805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa10c4c-c2c9-4eb3-a41d-c907852c12e2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.391996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 326/659] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:19.393602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa10c4c-c2c9-4eb3-a41d-c907852c12e2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:19.401532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb41274-7999-49a5-846b-edeac195fd01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:19.404991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb41274-7999-49a5-846b-edeac195fd01 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:19.441216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:19.536357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa10c4c-c2c9-4eb3-a41d-c907852c12e2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:19.552041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 334/660] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:19.552041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5b24f-5e47-4994-9bcb-c881c28c8627 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.552041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5b24f-5e47-4994-9bcb-c881c28c8627 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.552041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5b24f-5e47-4994-9bcb-c881c28c8627 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.552041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 335/661] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:19.552929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5b24f-5e47-4994-9bcb-c881c28c8627 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:19.584032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac7832-109a-4cf0-943f-89b877402a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:19.584695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7832-109a-4cf0-943f-89b877402a84 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.584941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7832-109a-4cf0-943f-89b877402a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.585236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7832-109a-4cf0-943f-89b877402a84 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.585990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac7832-109a-4cf0-943f-89b877402a84 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:19.586297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac7832-109a-4cf0-943f-89b877402a84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:19.587759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac7832-109a-4cf0-943f-89b877402a84 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:19.607382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 327/662] 10.222.0.43 () {56 vars in 1137 bytes} [Tue Sep 13 17:48:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:19.608697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06702273-e4ae-4e05-9a0d-bf5dc58fe1bc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.609029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06702273-e4ae-4e05-9a0d-bf5dc58fe1bc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.609252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06702273-e4ae-4e05-9a0d-bf5dc58fe1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.629168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-06702273-e4ae-4e05-9a0d-bf5dc58fe1bc None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:19.630033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:19.630412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-06702273-e4ae-4e05-9a0d-bf5dc58fe1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:19.684812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5b24f-5e47-4994-9bcb-c881c28c8627 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:19.722775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ac3ec-9997-4fed-9438-156d8cedc000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:19.723411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac3ec-9997-4fed-9438-156d8cedc000 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.723706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac3ec-9997-4fed-9438-156d8cedc000 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.723970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ac3ec-9997-4fed-9438-156d8cedc000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.736528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ac3ec-9997-4fed-9438-156d8cedc000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:19.738034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ac3ec-9997-4fed-9438-156d8cedc000 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:19.744160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 328/663] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:19.745698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06702273-e4ae-4e05-9a0d-bf5dc58fe1bc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4b2ac3ec-9997-4fed-9438-156d8cedc000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:19.754423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:19.755426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:19.756507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:19.756507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:19.756507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:19.763657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 336/664] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:19.763657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52c3bb-d434-4731-8094-944b8e6c2625 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.763657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52c3bb-d434-4731-8094-944b8e6c2625 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.763657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52c3bb-d434-4731-8094-944b8e6c2625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.793120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6f52c3bb-d434-4731-8094-944b8e6c2625 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:19.794992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6f52c3bb-d434-4731-8094-944b8e6c2625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:19.798826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:19.901730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06702273-e4ae-4e05-9a0d-bf5dc58fe1bc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:19.936458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 337/665] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:19.936458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52c3bb-d434-4731-8094-944b8e6c2625 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:19.966150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:19.967755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:19.968315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:19.968880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:19.989553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:19.993222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:20.023907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:20.025133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:20.026113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:20.026113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:20.026113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:20.026940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:20.027385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 329/666] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:20.028866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.123240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52c3bb-d434-4731-8094-944b8e6c2625 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:20.177984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:20.184723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:20.185761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:20.186252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:20.186735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:20.218122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e2400-50d2-413b-b5e6-4af7f2b01633 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:20.256146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:20.256801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:20.257089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:20.257356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:20.258028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:20.258326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:20.259761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:20.273161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 330/667] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:20.274045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.300519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:20.302198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:20.346249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:20.404860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55773b47-25dd-4c78-bfd0-e5dd695b7162 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:20.434325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 338/668] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:20.434325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.458452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:20.459302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:20.459742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:20.460171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:20.461274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:20.461753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:20.464129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:20.468153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:20.482737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 331/669] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:20.483834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.578414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c79b8-149c-4348-a258-3eafec39b45b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:20.626507 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46e7b8-060c-40cd-b316-cf582760246a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:20.632501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:20.633246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:20.633593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:20.633923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:20.644458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:20.650155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:20.655622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:20.657136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 339/670] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:20.658259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.673353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:20.674223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:20.674637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:20.675007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:20.687444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:20.693827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:20.695794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 332/671] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:20.697506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.766925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:20.777849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cecafd-4b08-4332-9968-89e12abebb7c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:20.813785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:20.814819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:20.814819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:20.815021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:20.823317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:20.825580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:20.829389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b873a-a8b8-408c-b94e-f75f2b55556e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:20.837437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:20.838861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 340/672] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:20.840278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.878904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:20.879540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:20.879823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:20.880118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:20.893994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:20.895523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:20.911734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:20.912997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:20.914186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:20.914186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:20.914186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:20.914358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:20.914431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 333/673] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:20.914431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:20.973184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20188fc5-30fb-4b30-a31e-df63e4345608 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.016817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.017480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.017762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.018036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.029337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.031842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.048670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e63f7-06b0-49f5-beae-a0d9553f5cb5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:21.049476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:21.050844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 341/674] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:21.053114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.062809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:21.094742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.095366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.095647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.095921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.096663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:21.096966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.098388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.107676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 334/675] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:21.108586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.214238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29e52c-207a-4cd3-a61c-a563b3e9636a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.224003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f3936-6a68-4ace-b4c0-92f24f19bfd4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.245405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:21.260531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.261152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.261448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.262082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.262795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:21.263116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.264637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.272016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.272958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.273355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.273690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.274457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:21.274869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.276375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.284833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 335/676] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:21.284833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.293622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 342/677] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:21.294498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.324253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:21.431689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82236f3e-ff2d-4164-a3ea-78c77f6769e5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.432539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daaad3-0afc-4dea-8169-ba7412dbaf9a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.474113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.474326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.474900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.475092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.475287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.475390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.475633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.475784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.476297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:21.476552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:21.476704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.476971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.478041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.478531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.491500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 336/678] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:21.492378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.499076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 343/679] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:21.499076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.547579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:21.619302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:21.634287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f100500-9224-46cd-84fc-ad8d9895232a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.636233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503b454-fac9-43c0-b0bf-0537c2fbf1b0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.675317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.676086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.676553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.676959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.677851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=6c9c5205f9344e8c8fcc85c80d6f504b) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:21.682401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.682401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.685840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.686501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.686768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.687041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.704960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 344/680] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/role_assignments?user.id=6c9c5205f9344e8c8fcc85c80d6f504b&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:21.704960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.718272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.721423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:21.751337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:21.752387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:21.753692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:21.753692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:21.753692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:21.753692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 337/681] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:21.755033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:21.862997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a8346-9612-4dd2-a73b-8fc58d97d5fa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.874739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:21.924059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:21.925135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:21.926820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:21.926820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:21.944945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e2881-83d7-4cdd-8015-b585bd2a7814 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:21.945165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:21.947822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:21.972273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:21.974508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:21.976664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:21.976664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:21.976664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:21.977175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 345/682] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:21.978382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.005597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.006378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.006786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.007188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.017081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.018955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:22.035640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:22.037930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 338/683] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:22.039891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.043543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:22.137488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df44fb3-b2fc-4c23-97f2-6816dd09165e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:22.170071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a735b-c08d-4b1c-86cc-47ec19fa178c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:22.183541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.184167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.184460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.184737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.185494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=a4b10367002d4f64a82b3e9eca00ca06) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:22.185801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.187269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.194580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 346/684] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:21 2022] GET /identity/v3/role_assignments?user.id=a4b10367002d4f64a82b3e9eca00ca06&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:22.195466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.243281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:22.244997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.246874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.247537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.248236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.250251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:22.251508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.254002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.276605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 339/685] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:22.278170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.384857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:22.411623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6e7dc-357c-4096-b4fb-8c91e4be0d8d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:22.423806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6354cec-f204-42fc-a154-8bed83dfc8eb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:22.472478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.473174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.473581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.474040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.474134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.474559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.474691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.475003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.475719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:22.476022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:22.476128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.476390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.478116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.480753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.494657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 340/686] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:22.495587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.504192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 347/687] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:22.505940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.612895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:22.627458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cfe2e-a3f5-423c-a304-f223c7ba8bac None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:22.641349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41ddcb-cdcc-46da-820e-c4100dd426e9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:22.654954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:22.676326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.677070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.677348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.677750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.692732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.695797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.703356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.704141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.704544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.704889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.705731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=2c282d6f237542e3b873100748759794) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:22.706115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.707685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:22.713860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:22.714864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:22.715794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:22.723019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 348/688] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/role_assignments?user.id=2c282d6f237542e3b873100748759794&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:22.723019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.723745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 341/689] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:22.725469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:22.894933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca489-2421-4a39-bffe-5f0dd396bec9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:22.943064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:22.944101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:22.944582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:22.945026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:22.946032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:22.946510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:22.948157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:22.977335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 342/690] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:22.980958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:23.024953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241eb796-83c8-481a-a5ad-840703d59fe7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:23.056838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:23.082978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:23.084232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:23.084538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:23.084811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:23.105366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:23.108132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:23.132246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:23.133387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:23.134345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:23.134898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 349/691] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:23.140920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:23.142587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f1da2-081e-4fe3-a19c-49f29a0978bb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:23.145544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:23.185050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:23.186061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:23.186570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:23.187055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:23.288818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:23.290348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:23.333050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed581c18-193a-4c3b-a131-c38099f8b0d9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:23.369910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:23.373488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:23.374152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:23.374871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:23.374871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:23.375494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:23.375860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:23.377371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:23.398003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 350/692] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:48:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:23.408089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:23.484711 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 343/693] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:23.490597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:23.502588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:23.549171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bb630-8067-4a24-8470-0ee3741a2ae5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:23.608124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:23.609057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:23.609471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:23.609896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:23.722998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93535c4-e8ee-4212-96cd-ac38466859b2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:23.769050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:23.771538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:23.824210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:23.824875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:23.825152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:23.825421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:23.842870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:23.844454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:23.846035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:23.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:23.886439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:23.888519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:23.888519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:23.888519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:23.888519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 344/694] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:23.898123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:23.925090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:24.021113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 351/695] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 17:48:24.029737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:24.177147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823f27c-ff1e-4b39-a4b0-17f208388bd9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:24.247091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:24.253716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.254166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.254624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.256116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:24.256686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:24.259151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:24.265711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12812ad2-f5d4-4d59-b47b-d3a03eef9d9f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:24.302675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 345/696] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:24.305933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:24.306952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.307456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.307926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.311348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:24.320149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:24.322194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:24.352904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:24.353998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:24.354970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:24.354970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:24.354970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:24.363716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 352/697] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:24.365997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:24.407211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:24.423945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:24.516231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba43eff-c4f2-4c89-ae51-aa4e85a45e07 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:24.581784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd35cb2-99cd-4ba9-8583-f55666b9636e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:24.591315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:24.591964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.592226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.592516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.603976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:24.605502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:24.622643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:24.624329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:24.626171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 346/698] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:24.630238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:24.674902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:24.675533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.675812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.676129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.676832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:24.677140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:24.680296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:24.705340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 353/699] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:24.712530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:24.849656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c93627-164b-4de4-a998-29941aaff2ad None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:24.877434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:24.879883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:24.917300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f1b8a-fcee-4bee-8772-48175dac4f3b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:24.934581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4076b-9eb5-4b4c-be59-8a9c4ddf9c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:24.935204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4076b-9eb5-4b4c-be59-8a9c4ddf9c20 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.935483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4076b-9eb5-4b4c-be59-8a9c4ddf9c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.935744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4076b-9eb5-4b4c-be59-8a9c4ddf9c20 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.936473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4076b-9eb5-4b4c-be59-8a9c4ddf9c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:24.936781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4076b-9eb5-4b4c-be59-8a9c4ddf9c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:24.938233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4076b-9eb5-4b4c-be59-8a9c4ddf9c20 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:24.951784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 347/700] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:24.953135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d765a-1131-45d6-a612-e13a309fff22 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.953415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d765a-1131-45d6-a612-e13a309fff22 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.953683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d765a-1131-45d6-a612-e13a309fff22 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.955632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 348/701] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:24.961432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac4b7-52e5-4fc0-9187-5035d35b2c8b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.961963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac4b7-52e5-4fc0-9187-5035d35b2c8b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.962370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac4b7-52e5-4fc0-9187-5035d35b2c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.969609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:24.970474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:24.971679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:24.971679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:24.983005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e37ac4b7-52e5-4fc0-9187-5035d35b2c8b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:24.983224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:24.984681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e37ac4b7-52e5-4fc0-9187-5035d35b2c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:24.984816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:24.998127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:24.999106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:25.000065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 354/702] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:25.000834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:25.080465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 349/703] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:48:25.080465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ac4b7-52e5-4fc0-9187-5035d35b2c8b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:25.266886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f48f-8734-4e3f-8e67-d7de83833825 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:25.317170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ac4b7-52e5-4fc0-9187-5035d35b2c8b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:25.324813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:25.385421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:25.386452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:25.387307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:25.387781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:25.389276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1ec763f1afac469196a5e897587813b8) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:25.389809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:25.393545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:25.397891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:25.415128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 355/704] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:24 2022] GET /identity/v3/role_assignments?user.id=1ec763f1afac469196a5e897587813b8&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 608 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:25.416238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:25.417139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:25.417630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:25.418103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:25.419063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:25.419613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:25.421450 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:25.438160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:25.439193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 350/705] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:25.457005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:25.609305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8caf45-f9b9-42ee-8b1d-a8318cfc2fb5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:25.660977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a45f08-452b-433a-88a0-3cc05a9c8ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:25.662019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a45f08-452b-433a-88a0-3cc05a9c8ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:25.662656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a45f08-452b-433a-88a0-3cc05a9c8ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:25.663221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a45f08-452b-433a-88a0-3cc05a9c8ea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:25.664188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a45f08-452b-433a-88a0-3cc05a9c8ea1 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=90f40fe8f20f4a5bbd201debda6a97a5) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:25.665018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a45f08-452b-433a-88a0-3cc05a9c8ea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:25.667329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a45f08-452b-433a-88a0-3cc05a9c8ea1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:25.674604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 351/706] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:25 2022] GET /identity/v3/role_assignments?user.id=90f40fe8f20f4a5bbd201debda6a97a5&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:25.680960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3b0bd-eb54-4f40-ba9c-6977c6965723 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:25.681261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3b0bd-eb54-4f40-ba9c-6977c6965723 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:25.681541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3b0bd-eb54-4f40-ba9c-6977c6965723 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:25.685366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d781418-88bf-4964-ade7-4ff6b3b65bcb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:25.687543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 352/707] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:25.695459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627a42-e19d-417f-97d2-edd445f7f016 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:25.695806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627a42-e19d-417f-97d2-edd445f7f016 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:25.696085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627a42-e19d-417f-97d2-edd445f7f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:25.721196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-22627a42-e19d-417f-97d2-edd445f7f016 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:25.722459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-22627a42-e19d-417f-97d2-edd445f7f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:25.759450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:25.760925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:25.761562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:25.762740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:25.763713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:25.765177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:25.768669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:25.785794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:25.787041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:25.787954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:25.791067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 356/708] 10.222.0.43 () {56 vars in 1129 bytes} [Tue Sep 13 17:48:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:25.799139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:25.838158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 353/709] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 17:48:25.890965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:25.939578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d110-8b96-4ad1-ba73-17a2fc43bae8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:25.978362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cadad-fd93-4c56-a940-a1d1a0ac620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:25.979000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cadad-fd93-4c56-a940-a1d1a0ac620b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:25.979279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cadad-fd93-4c56-a940-a1d1a0ac620b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:25.979614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cadad-fd93-4c56-a940-a1d1a0ac620b None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:25.980279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cadad-fd93-4c56-a940-a1d1a0ac620b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:25.980654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cadad-fd93-4c56-a940-a1d1a0ac620b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:25.982082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cadad-fd93-4c56-a940-a1d1a0ac620b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:25.991130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 357/710] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 13 17:48:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:25.994310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627a42-e19d-417f-97d2-edd445f7f016 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:26.142636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c77393-ce8e-4669-8855-2a0c719f10e2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:26.143037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c77393-ce8e-4669-8855-2a0c719f10e2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:26.143447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c77393-ce8e-4669-8855-2a0c719f10e2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:26.146384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 358/711] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:26.158940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c33b-619d-4810-8317-3dc60ddb62cb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:26.159892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c33b-619d-4810-8317-3dc60ddb62cb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:26.160382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770c33b-619d-4810-8317-3dc60ddb62cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:26.172492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627a42-e19d-417f-97d2-edd445f7f016 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:26.193998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8770c33b-619d-4810-8317-3dc60ddb62cb None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:26.196256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8770c33b-619d-4810-8317-3dc60ddb62cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:26.250151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:26.250918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:26.251255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:26.251683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:26.252614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:26.253080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:26.256631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:26.267687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 354/712] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 13 17:48:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:26.268929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:26.314031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:26.334671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 359/713] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:26.487934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5766e-364e-4ee3-b1db-539c0d1ed6a6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:26.541497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:26.542156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:26.542432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:26.542702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] PATH_INFO: `/v3/services/network` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:26.543290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:26.544741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:26.557727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:26.558790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:26.559628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 355/714] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:26 2022] GET /identity/v3/services/network => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:26.567211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c33b-619d-4810-8317-3dc60ddb62cb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:26.649788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:26.741678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:26.747651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770c33b-619d-4810-8317-3dc60ddb62cb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:26.803088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:26.804154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:26.804710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:26.805184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:26.806199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] RBAC: Adding query filter params (name=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:26.806841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:26.808264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:26.823254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 360/715] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:48:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:26.828998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:26.884848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:26.917078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279fd27-090b-4fb7-80b1-31cc99c6ef75 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:26.999188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:26.999751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:27.000024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:27.000293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:27.022901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:27.025781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:27.045289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c235e17-f2be-4b71-a8ea-3efa043c693d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:27.057869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:27.059325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:27.060225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:27.060225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:27.060225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:27.065093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 356/716] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:27.069614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:27.072641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:27.104913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:27.105828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:27.106248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:27.106650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:27.107644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] RBAC: Adding query filter params (type=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:27.108104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:27.110393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:27.133828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 361/717] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:48:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:27.262790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0329-d352-4e03-8336-dfa0997efea6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:27.321408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76586e9d-df8b-43dc-812c-dd0a1710351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:27.322353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76586e9d-df8b-43dc-812c-dd0a1710351c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:27.322805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76586e9d-df8b-43dc-812c-dd0a1710351c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:27.323225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76586e9d-df8b-43dc-812c-dd0a1710351c None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:27.324077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76586e9d-df8b-43dc-812c-dd0a1710351c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:27.324600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76586e9d-df8b-43dc-812c-dd0a1710351c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:27.326230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76586e9d-df8b-43dc-812c-dd0a1710351c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:27.329603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:27.347468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 357/718] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:27.357032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:27.569108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4bbf9-fabf-4bd2-a4ba-6d37aef0c65a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:27.641186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e3224-dd9e-42e9-9e1d-fff503b67047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:27.642017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e3224-dd9e-42e9-9e1d-fff503b67047 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:27.642300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e3224-dd9e-42e9-9e1d-fff503b67047 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:27.642655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e3224-dd9e-42e9-9e1d-fff503b67047 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:27.643597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e3224-dd9e-42e9-9e1d-fff503b67047 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:27.645601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e3224-dd9e-42e9-9e1d-fff503b67047 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:27.693008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 362/719] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:48:27 2022] POST /identity/v3/users => generated 254 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:28.463718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842289c4-9f05-4635-bb19-c6290c1cdbba None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:28.464593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842289c4-9f05-4635-bb19-c6290c1cdbba None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:28.464731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842289c4-9f05-4635-bb19-c6290c1cdbba None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:28.467587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 358/720] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:28.476375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de8b1-cd29-477a-9f54-5a0e48cb0e19 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:28.477532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de8b1-cd29-477a-9f54-5a0e48cb0e19 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:28.477532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de8b1-cd29-477a-9f54-5a0e48cb0e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:28.510237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4a4de8b1-cd29-477a-9f54-5a0e48cb0e19 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:28.512748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4a4de8b1-cd29-477a-9f54-5a0e48cb0e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:28.588447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:28.669893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 363/721] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:28.779419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c03d9-0900-4aa9-aff8-60f02cb7823e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:28.780379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c03d9-0900-4aa9-aff8-60f02cb7823e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:28.781167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c03d9-0900-4aa9-aff8-60f02cb7823e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:28.784351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 359/722] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:28.790650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6bf2-8999-42a5-bd08-f95e05ae6a5b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:28.791556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6bf2-8999-42a5-bd08-f95e05ae6a5b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:28.792317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a6bf2-8999-42a5-bd08-f95e05ae6a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:28.830498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ae5a6bf2-8999-42a5-bd08-f95e05ae6a5b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:28.833047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ae5a6bf2-8999-42a5-bd08-f95e05ae6a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:28.990372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 364/723] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:29.207618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c03d9-0900-4aa9-aff8-60f02cb7823e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:29.418198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c03d9-0900-4aa9-aff8-60f02cb7823e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:29.449491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a6bf2-8999-42a5-bd08-f95e05ae6a5b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:29.521468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:29.523768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:29.524799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:29.525548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:29.553286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:29.562103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:29.564106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:29.566073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 360/724] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:29.569330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:29.680264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:29.687021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a6bf2-8999-42a5-bd08-f95e05ae6a5b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:29.731490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:29.740291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:29.743181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:29.744760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:29.745209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:29.779434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:29.784812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:29.785966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:29.787012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:29.794595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 365/725] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:29.801084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:29.864541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce15b53-6e41-407a-a857-773c84586a2d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:29.968343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:29.969396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:29.969882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:29.970349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:29.971237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:29.971753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:29.973398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:29.992653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 361/726] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:29.998116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:30.062514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d7fdb-d2b6-4707-a67e-00e09d6e84d3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:30.110792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c274ad-2731-47a5-90e7-bbedf948f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:30.111818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c274ad-2731-47a5-90e7-bbedf948f2dd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.112295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c274ad-2731-47a5-90e7-bbedf948f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.112816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c274ad-2731-47a5-90e7-bbedf948f2dd None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.113910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c274ad-2731-47a5-90e7-bbedf948f2dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:30.114473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c274ad-2731-47a5-90e7-bbedf948f2dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:30.116874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c274ad-2731-47a5-90e7-bbedf948f2dd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:30.127141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 366/727] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:30.128949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888bfc6-60ea-4802-b0be-56d7ce1589a1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.129448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888bfc6-60ea-4802-b0be-56d7ce1589a1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.129978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888bfc6-60ea-4802-b0be-56d7ce1589a1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.131862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 367/728] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:30.133124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888bfc6-60ea-4802-b0be-56d7ce1589a1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:30.151007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5018a6-59b7-46fb-8c8b-67ac1710a0bf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:30.163075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:30.190274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:30.224069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fed9d-5e87-42e6-ae3d-36798d209d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:30.225146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fed9d-5e87-42e6-ae3d-36798d209d60 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.225622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fed9d-5e87-42e6-ae3d-36798d209d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.226434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fed9d-5e87-42e6-ae3d-36798d209d60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.247898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fed9d-5e87-42e6-ae3d-36798d209d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:30.250922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fed9d-5e87-42e6-ae3d-36798d209d60 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:30.255178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888bfc6-60ea-4802-b0be-56d7ce1589a1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1e7fed9d-5e87-42e6-ae3d-36798d209d60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:30.284006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:30.285260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:30.286144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:30.286144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:30.286144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:30.286628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 362/729] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:30.288987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687ac9-4006-4e3d-a134-c8f6150c9932 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.289756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687ac9-4006-4e3d-a134-c8f6150c9932 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.290216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687ac9-4006-4e3d-a134-c8f6150c9932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.300037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:30.301142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.301636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.302104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.326990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f0687ac9-4006-4e3d-a134-c8f6150c9932 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:30.328789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f0687ac9-4006-4e3d-a134-c8f6150c9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:30.331707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:30.336588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:30.368062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:30.369999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:30.371773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:30.371773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:30.371773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:30.380565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 368/730] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:30.383779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:30.486062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 363/731] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:30.487755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0687ac9-4006-4e3d-a134-c8f6150c9932 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:30.564342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:30.617896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950160d1-f327-4e0e-bf5e-8a22fb6dd6c0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:30.702622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399098a8-faaf-4767-9b62-291513189d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:30.703688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399098a8-faaf-4767-9b62-291513189d7c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.704159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399098a8-faaf-4767-9b62-291513189d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.704759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399098a8-faaf-4767-9b62-291513189d7c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.706307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399098a8-faaf-4767-9b62-291513189d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:30.706691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399098a8-faaf-4767-9b62-291513189d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:30.709629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399098a8-faaf-4767-9b62-291513189d7c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:30.736223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 369/732] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:30.738644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:30.739360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f86f6-9abe-4035-adc8-3f848ed1e64a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.739670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f86f6-9abe-4035-adc8-3f848ed1e64a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.740173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f86f6-9abe-4035-adc8-3f848ed1e64a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.743966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 370/733] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:30.745215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f86f6-9abe-4035-adc8-3f848ed1e64a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:30.751669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0687ac9-4006-4e3d-a134-c8f6150c9932 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:30.820152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55115745-a312-4eee-ac7c-de0a95acf9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:30.821829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115745-a312-4eee-ac7c-de0a95acf9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.822765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115745-a312-4eee-ac7c-de0a95acf9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.823701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55115745-a312-4eee-ac7c-de0a95acf9c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.825348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115745-a312-4eee-ac7c-de0a95acf9c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:30.826222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115745-a312-4eee-ac7c-de0a95acf9c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:30.829162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55115745-a312-4eee-ac7c-de0a95acf9c9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:30.854461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 364/734] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:30.857310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a4bd-ce0a-463c-a966-d3064b41d189 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.858099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a4bd-ce0a-463c-a966-d3064b41d189 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.858812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a4bd-ce0a-463c-a966-d3064b41d189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.894625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2529a4bd-ce0a-463c-a966-d3064b41d189 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:30.897095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2529a4bd-ce0a-463c-a966-d3064b41d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:30.920479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f86f6-9abe-4035-adc8-3f848ed1e64a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:30.974821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:30.975861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:30.976294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:30.976777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:30.980328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:30.991374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:30.993127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:31.009572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:31.011487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 371/735] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:31.014215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:31.024490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 365/736] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:31.026353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529a4bd-ce0a-463c-a966-d3064b41d189 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:31.195057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab36505-3176-45b1-9756-7c224d2058e7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:31.209960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:31.227727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529a4bd-ce0a-463c-a966-d3064b41d189 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:31.249240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:31.250198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:31.250753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:31.251214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:31.252343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:31.255048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:31.283302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 372/737] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:48:31 2022] POST /identity/v3/services => generated 251 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:31.284311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:31.310606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca387dbd-b3e3-4284-b34a-fb3555d6b897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:31.311693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca387dbd-b3e3-4284-b34a-fb3555d6b897 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:31.312168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca387dbd-b3e3-4284-b34a-fb3555d6b897 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:31.313402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca387dbd-b3e3-4284-b34a-fb3555d6b897 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:31.328033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca387dbd-b3e3-4284-b34a-fb3555d6b897 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:31.330961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca387dbd-b3e3-4284-b34a-fb3555d6b897 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ca387dbd-b3e3-4284-b34a-fb3555d6b897 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:31.355008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:31.357211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:31.359308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 366/738] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:48:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:31.360594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9351-a2ae-4296-9cc7-3869a0853cb1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:31.360594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9351-a2ae-4296-9cc7-3869a0853cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:31.361051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9351-a2ae-4296-9cc7-3869a0853cb1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:31.369423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 367/739] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:31.369423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9351-a2ae-4296-9cc7-3869a0853cb1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:31.409908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:31.493372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67703e-a041-4a53-aa46-c550e4932fd4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:31.543556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcaf82-48f2-4955-857f-f32430b67d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:31.544346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcaf82-48f2-4955-857f-f32430b67d10 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:31.544723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcaf82-48f2-4955-857f-f32430b67d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:31.545061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcaf82-48f2-4955-857f-f32430b67d10 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:31.545888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcaf82-48f2-4955-857f-f32430b67d10 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:31.546258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcaf82-48f2-4955-857f-f32430b67d10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:31.547852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcaf82-48f2-4955-857f-f32430b67d10 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:31.564444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e9351-a2ae-4296-9cc7-3869a0853cb1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:31.568314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 373/740] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 13 17:48:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:31.570094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7adc83-4316-4a20-9620-d3bfb6d73c57 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:31.570368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7adc83-4316-4a20-9620-d3bfb6d73c57 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:31.570759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7adc83-4316-4a20-9620-d3bfb6d73c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:31.592851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6f7adc83-4316-4a20-9620-d3bfb6d73c57 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:31.594845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6f7adc83-4316-4a20-9620-d3bfb6d73c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:31.632222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:31.633340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:31.633830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:31.634314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:31.635723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:31.636258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:31.639329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:31.679923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 368/741] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:48:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:31.679923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:31.688258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:31.733069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 374/742] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 17:48:31.734049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7adc83-4316-4a20-9620-d3bfb6d73c57 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:31.792594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:31.873501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686dff-d878-413d-8e82-9d86263457e5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:31.912824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7adc83-4316-4a20-9620-d3bfb6d73c57 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:31.920487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:31.920487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:31.920487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:31.921051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:31.921361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:31.922947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:31.953046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 369/743] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:48:31 2022] POST /identity/v3/services => generated 240 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:31.954287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:32.020552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:32.022394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:32.023348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:32.024138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:32.081846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:32.172346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fdb6a-4ff0-4fcd-a646-12d5daab0054 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:32.201079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:32.203385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:32.252157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:32.253223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:32.253501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:32.253764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:32.269135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:32.270774 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:32.289144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:32.290957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:32.293477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:32.293477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:32.293477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:32.296715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 370/744] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:32.296715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:32.297563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:32.430241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 375/745] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:48:32.431117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:32.468915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d828e-404f-4465-9baf-5e0322986c12 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:32.494370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:32.517291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:32.517994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:32.518281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:32.518544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:32.532770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:32.538445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:32.539434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 371/746] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:32.540704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:32.578787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fbd0b-7583-4515-91ef-36e85f28544c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:32.627212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:32.627845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:32.628124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:32.628404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:32.629109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:32.629383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:32.630855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:32.641409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:32.648329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 376/747] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:32.654680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:32.708016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077258a-e4ba-4163-99cd-81d7c9e91b43 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:32.753439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:32.754245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:32.754537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:32.754791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:32.769098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:32.770615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:32.789301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:32.791504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:32.793445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:32.793445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:32.793445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:32.793445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 372/748] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:32.793445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:32.834712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:32.877443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5498432-8777-422f-85cc-2d3c58cfe31f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:32.920081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:32.920873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:32.921101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:32.921325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:32.922025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:32.922325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:32.923763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:32.934342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 377/749] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:32.935353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:33.009224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:33.009637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf43499-bd03-4699-81df-2d970fb8312b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:33.057022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:33.057848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:33.058218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:33.058588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:33.067543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:33.069955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:33.084195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:33.086131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:33.088128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:33.092354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 373/750] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:33.093497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:33.108412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e00ed-fbe4-45ae-a6d6-f01ba6640e90 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:33.173893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:33.175237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:33.175914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:33.176986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:33.179088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:33.179906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:33.184269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:33.213241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 378/751] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:33.215143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:33.231986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:33.263145 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8225e-9f93-4c10-bbcd-18b73c7b69e7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:33.358795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:33.359515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae6ae-7d1f-4fbe-8ebd-7fb4fe862f4c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:33.360524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:33.361244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:33.362392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:33.388618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:33.395137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:33.401242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:33.407124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:33.407752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:33.408028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:33.408294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:33.409064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:33.409371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:33.410855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:33.429971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 379/752] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:33.431081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:33.438641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:33.441176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:33.442887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:33.442887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:33.442887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:33.442887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 374/753] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:33.442887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:33.597354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9b3f4-adc5-4708-bb0f-2dd34610c6d7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:33.635257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d4de8-5bcb-4711-ad77-0f966dc3512f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:33.650422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:33.651064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:33.651340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:33.651668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:33.659806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:33.661327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:33.674994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:33.676815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:33.679089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:33.680833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 380/754] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:33.681788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:33.709952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:33.710627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:33.710903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:33.711159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/90f40fe8f20f4a5bbd201debda6a97a5/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:33.713216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=90f40fe8f20f4a5bbd201debda6a97a5, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:33.714691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:33.728449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:33.740740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 375/755] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:33 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/90f40fe8f20f4a5bbd201debda6a97a5/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:33.741928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:33.773389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:33.912789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4b61f-c74c-49cf-93e3-0bcbfc44cea4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:33.985500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978fad8-cff0-42be-8b63-a0b98f897560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:33.986147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978fad8-cff0-42be-8b63-a0b98f897560 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:33.986428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978fad8-cff0-42be-8b63-a0b98f897560 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:33.986707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978fad8-cff0-42be-8b63-a0b98f897560 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:33.987388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978fad8-cff0-42be-8b63-a0b98f897560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:33.987696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978fad8-cff0-42be-8b63-a0b98f897560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:33.989160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978fad8-cff0-42be-8b63-a0b98f897560 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:33.990906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e005f05-68e5-453f-aed5-d436cd40c1ad None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:34.012050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 381/756] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:34.014682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2989ec-c90a-42c3-8534-5ea55dce1f26 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:34.015435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2989ec-c90a-42c3-8534-5ea55dce1f26 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:34.016250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2989ec-c90a-42c3-8534-5ea55dce1f26 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:34.019370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 382/757] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:34.021135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2989ec-c90a-42c3-8534-5ea55dce1f26 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:34.056993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff06e9c-708f-47a0-be19-675f0b8a8dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:34.058965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff06e9c-708f-47a0-be19-675f0b8a8dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:34.059712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff06e9c-708f-47a0-be19-675f0b8a8dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:34.060503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff06e9c-708f-47a0-be19-675f0b8a8dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:34.062037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff06e9c-708f-47a0-be19-675f0b8a8dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:34.062844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff06e9c-708f-47a0-be19-675f0b8a8dd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:34.064820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff06e9c-708f-47a0-be19-675f0b8a8dd4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:34.085759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 376/758] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:34.088341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f70f0-609b-47a4-ac0d-6d611ce476f3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:34.088999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f70f0-609b-47a4-ac0d-6d611ce476f3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:34.089740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f70f0-609b-47a4-ac0d-6d611ce476f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:34.120978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f46f70f0-609b-47a4-ac0d-6d611ce476f3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:34.122438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f46f70f0-609b-47a4-ac0d-6d611ce476f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:34.188402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:34.194267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2989ec-c90a-42c3-8534-5ea55dce1f26 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:34.209923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:34.224577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 377/759] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:34.225422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f70f0-609b-47a4-ac0d-6d611ce476f3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:34.271919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:34.273958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:34.274755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:34.275515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:34.292802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:34.297559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:34.328301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:34.330129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:34.331921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:34.337603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 383/760] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:48:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:34.346321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:34.444096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f70f0-609b-47a4-ac0d-6d611ce476f3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:34.525105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:34.525728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:34.526002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:34.526267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:34.527011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1ec763f1afac469196a5e897587813b8) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:34.527314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:34.528957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:34.538870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 378/761] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:34 2022] GET /identity/v3/role_assignments?user.id=1ec763f1afac469196a5e897587813b8&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:34.540917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:34.581085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc8f7a-997a-4d8c-810d-e5b09c5f94b8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:34.674077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:34.675119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:34.675586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:34.676053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:34.686743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:34.687342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:34.690108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:34.696896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:34.729973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 384/762] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 13 17:48:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:34.739415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:34.802903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:34.818357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73539036-2b8e-4a6e-b219-20a1dc634ef5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:34.861677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:34.862335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:34.862725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:34.863001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:34.863607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:34.865152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:34.882804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:34.883869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:34.884752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:34.886074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 379/763] 10.222.0.43 () {56 vars in 1129 bytes} [Tue Sep 13 17:48:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:34.894787 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:35.000124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78a47a-6c5c-4cbb-807f-4d2673a9e10d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:35.065281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94624562-150c-4ed5-a2d0-7c0fecc60cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:35.066306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624562-150c-4ed5-a2d0-7c0fecc60cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.066788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624562-150c-4ed5-a2d0-7c0fecc60cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.067310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94624562-150c-4ed5-a2d0-7c0fecc60cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.135689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968715a3-9b91-46c3-9dd0-88650a911305 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:35.149433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94624562-150c-4ed5-a2d0-7c0fecc60cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:35.151261 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94624562-150c-4ed5-a2d0-7c0fecc60cd7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:35.173817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:35.176828 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:35.216876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccffb1-f89a-44f4-89a7-0dc5f555e914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:35.217863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccffb1-f89a-44f4-89a7-0dc5f555e914 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.218335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccffb1-f89a-44f4-89a7-0dc5f555e914 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.218803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccffb1-f89a-44f4-89a7-0dc5f555e914 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.220739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccffb1-f89a-44f4-89a7-0dc5f555e914 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:35.221642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccffb1-f89a-44f4-89a7-0dc5f555e914 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:35.225125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccffb1-f89a-44f4-89a7-0dc5f555e914 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:35.236483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 380/764] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 13 17:48:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:35.238225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634314d7-46aa-4f27-a11e-1ebad1e98b34 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.238716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634314d7-46aa-4f27-a11e-1ebad1e98b34 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.239189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634314d7-46aa-4f27-a11e-1ebad1e98b34 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.245189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 381/765] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:35.245189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634314d7-46aa-4f27-a11e-1ebad1e98b34 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:35.292817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 385/766] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 17:48:35.294755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cce54a-3de5-4095-b34a-e7836bd5fb02 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.295293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cce54a-3de5-4095-b34a-e7836bd5fb02 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.295868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cce54a-3de5-4095-b34a-e7836bd5fb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.319545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e5cce54a-3de5-4095-b34a-e7836bd5fb02 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:35.320989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e5cce54a-3de5-4095-b34a-e7836bd5fb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:35.416046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 386/767] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:35.417402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cce54a-3de5-4095-b34a-e7836bd5fb02 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:35.419431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634314d7-46aa-4f27-a11e-1ebad1e98b34 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:35.469663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:35.470521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.470893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.471255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.472208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:35.472635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:35.474875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:35.475752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:35.482487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 382/768] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 13 17:48:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:35.493585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:35.544462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cce54a-3de5-4095-b34a-e7836bd5fb02 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:35.587002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:35.587917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.588394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.588823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.596706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:35.602424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:35.604137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:35.623822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:35.624976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:35.625891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:35.625891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:35.625891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:35.626933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 387/769] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:35.634713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:35.724501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ffb36-fc9a-483e-9297-1b2004bde74f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:35.803300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:35.803815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.804087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.804361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.805108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] RBAC: Adding query filter params (interface=public, service_id=90c4b2d6c30c4675a0cd404edc4d8faa, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:35.805419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:35.806864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:35.829051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 383/770] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:48:35 2022] GET /identity/v3/endpoints?service_id=90c4b2d6c30c4675a0cd404edc4d8faa&interface=public®ion_id=RegionOne => generated 194 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:35.836172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:35.848014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:35.853930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4e288-0557-4a27-8676-dfd137db2108 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:35.900455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:35.901183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:35.901499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:35.901832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:35.902502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:35.902806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:35.904526 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:35.920751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 388/771] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:35.921649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:35.958158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16cdc-4ea4-4ec3-914a-099a6633d0bc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:35.970849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:35.999090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:35.999727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.000004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.000277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.000951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:36.002456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:36.008904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 384/772] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:48:35 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:36.009772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:36.056659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee8eec-068d-4d03-8643-c97a00e06d51 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:36.127763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:36.135891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.136327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.136975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] PATH_INFO: `/v3/services/network` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.140463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:36.140463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:36.157539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:36.159574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:36.161308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 389/773] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:35 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:36.165223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:36.222516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e002728-13bf-4d64-8754-1d7864d6242f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:36.233632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:36.325155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d324b-afd1-4e8d-9736-952e91f379dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:36.326737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d324b-afd1-4e8d-9736-952e91f379dd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.327320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d324b-afd1-4e8d-9736-952e91f379dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.327896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d324b-afd1-4e8d-9736-952e91f379dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.341718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d324b-afd1-4e8d-9736-952e91f379dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:36.343259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d324b-afd1-4e8d-9736-952e91f379dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-699d324b-afd1-4e8d-9736-952e91f379dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:36.358913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:36.361093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 385/774] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173813-f690-4c1c-8e6f-d3dc7b478561 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.362909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173813-f690-4c1c-8e6f-d3dc7b478561 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.364102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173813-f690-4c1c-8e6f-d3dc7b478561 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.364102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 386/775] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:36.368970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173813-f690-4c1c-8e6f-d3dc7b478561 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:36.377068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4b616-48e2-4c7c-a171-3e7ca1a4a6ec None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:36.420509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe2cd1-2c6e-4edf-9a88-f8aedcf381cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:36.420509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2cd1-2c6e-4edf-9a88-f8aedcf381cc None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.420509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2cd1-2c6e-4edf-9a88-f8aedcf381cc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.420509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2cd1-2c6e-4edf-9a88-f8aedcf381cc None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.421172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe2cd1-2c6e-4edf-9a88-f8aedcf381cc None admin] RBAC: Adding query filter params (name=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:36.421172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe2cd1-2c6e-4edf-9a88-f8aedcf381cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:36.422347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe2cd1-2c6e-4edf-9a88-f8aedcf381cc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:36.428973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 390/776] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:48:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:36.430374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12270-3de7-4b36-819a-37a69cdd9244 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.430670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12270-3de7-4b36-819a-37a69cdd9244 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.430936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12270-3de7-4b36-819a-37a69cdd9244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.441550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:36.453468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-faf12270-3de7-4b36-819a-37a69cdd9244 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:36.454709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-faf12270-3de7-4b36-819a-37a69cdd9244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:36.540579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 391/777] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:36.544062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12270-3de7-4b36-819a-37a69cdd9244 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:36.614598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173813-f690-4c1c-8e6f-d3dc7b478561 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:36.620831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:36.694284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:36.694921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.695233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.695476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.696179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:36.696517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:36.698014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:36.742952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 387/778] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:36.757589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:36.831886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12270-3de7-4b36-819a-37a69cdd9244 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:36.878617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:36.879143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:36.879422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:36.879695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:36.880348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] RBAC: Adding query filter params (type=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:36.880749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:36.882228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:36.889189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 392/779] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:48:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:36.890090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:37.009693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:37.025267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51277f-2852-4a3c-abba-6be317498cf9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:37.065821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:37.101253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:37.102572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.103210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.103783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.103938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e041b9-c574-4591-a62e-c26d09d0e0ae None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:37.105382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1e6a922a2aea4d3ea1feadbd25cb32cf) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:37.106032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:37.110396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:37.124475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 388/780] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:36 2022] GET /identity/v3/role_assignments?user.id=1e6a922a2aea4d3ea1feadbd25cb32cf&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 259 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:37.127244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:37.175488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:37.176619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.177173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.177736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.198379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:37.209288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:37.211373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:37.213127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 393/781] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:37.217090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:37.364296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a4c85-5b22-44b3-96b4-b319251ee10b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:37.400082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb84b9-54fe-426d-a274-caeafd164ed1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:37.437270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:37.438467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.439097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.439678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.441221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] RBAC: Adding query filter params (interface=public, service_id=598d0a92605f4a9db8409fb2ccab23e7, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:37.441912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:37.447113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:37.455521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 389/782] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:48:37 2022] GET /identity/v3/endpoints?service_id=598d0a92605f4a9db8409fb2ccab23e7&interface=public®ion_id=RegionOne => generated 194 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:37.457275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:37.458101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.458503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.458938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.459816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:37.460251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:37.461838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:37.468463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:37.483016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 394/783] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:37.484346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:37.493430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:37.522944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:37.630148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698e7cc-07ae-47bf-ac55-7088f598d880 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:37.652343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20bf47-45e3-4bac-822c-9eac22c3b11f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:37.687261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547e3fb-79c0-4677-b14e-a528bd570aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:37.688636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547e3fb-79c0-4677-b14e-a528bd570aec None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.689194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547e3fb-79c0-4677-b14e-a528bd570aec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.689606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547e3fb-79c0-4677-b14e-a528bd570aec None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.690422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547e3fb-79c0-4677-b14e-a528bd570aec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:37.692139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547e3fb-79c0-4677-b14e-a528bd570aec None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:37.700301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e874254-d484-42d8-9857-59403497f808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:37.701025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874254-d484-42d8-9857-59403497f808 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.701307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874254-d484-42d8-9857-59403497f808 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.701632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e874254-d484-42d8-9857-59403497f808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.704757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 390/784] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:48:37 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:37.706738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5188b2-f7ed-4ee8-a1f3-c0e89c7d4ad9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.707120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5188b2-f7ed-4ee8-a1f3-c0e89c7d4ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.707486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5188b2-f7ed-4ee8-a1f3-c0e89c7d4ad9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.714121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 391/785] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:37.714976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e874254-d484-42d8-9857-59403497f808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:37.721124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e874254-d484-42d8-9857-59403497f808 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:37.722027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a31735-e48f-4bf5-aec9-a725c423204a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.722358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a31735-e48f-4bf5-aec9-a725c423204a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.722635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a31735-e48f-4bf5-aec9-a725c423204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5e874254-d484-42d8-9857-59403497f808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:37.738818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:37.739863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:37.742630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:37.742630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:37.742630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:37.742630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 395/786] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:37.746464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e874254-d484-42d8-9857-59403497f808 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:37.798532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e2a31735-e48f-4bf5-aec9-a725c423204a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:37.800798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2a31735-e48f-4bf5-aec9-a725c423204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:37.819530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:37.876614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e874254-d484-42d8-9857-59403497f808 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:37.955326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 392/787] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:37.970179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:37.970831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:37.971107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:37.971377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:37.972059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:37.972360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:37.973835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:37.998517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 396/788] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:38.004632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a31735-e48f-4bf5-aec9-a725c423204a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:38.037508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:38.130919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a31735-e48f-4bf5-aec9-a725c423204a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:38.170051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:38.170704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:38.170976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:38.171240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:38.179544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:38.181062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:38.198225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:38.199244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:38.200190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 393/789] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:48:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:38.209765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:38.329552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:38.398036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66b04f-bf3d-4fee-8c50-3280e2163a9d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:38.438366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:38.438919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:38.463024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f20f5-022d-4f4b-b286-16853609f395 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:38.505312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:38.506417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:38.507033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:38.507608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:38.521395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:38.521608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:38.522442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:38.522867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:38.523281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:38.524559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:38.525041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:38.526995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:38.527969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:38.530043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:38.530701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:38.530785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 394/790] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:38.535485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:38.597803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 397/791] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:48:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:38.606931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:38.729276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70613eac-61b0-4101-9c4b-59325f43aa26 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:38.733186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d10dba-2bfb-446c-acba-7c4f75d6cff3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:38.784702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:38.785339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:38.785615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:38.785878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:38.799161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:38.800706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:38.801514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:38.802265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:38.803827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:38.804642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:38.807516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:38.812031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:38.826645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 395/792] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:38.836567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:38.850321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:38.934777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:38.938025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:39.122684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62315e43-b52f-41f3-8591-32a4d57c0e52 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:39.202280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066ed0d-edb9-4f4f-b770-6f6d50534cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:39.206996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ed0d-edb9-4f4f-b770-6f6d50534cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:39.207762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ed0d-edb9-4f4f-b770-6f6d50534cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:39.215392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ed0d-edb9-4f4f-b770-6f6d50534cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:39.215654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 398/793] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:48:39.218291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:39.235989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066ed0d-edb9-4f4f-b770-6f6d50534cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:39.239256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066ed0d-edb9-4f4f-b770-6f6d50534cb7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3066ed0d-edb9-4f4f-b770-6f6d50534cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:39.269502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:39.271227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:39.273016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:39.273016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:39.273016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:39.277003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 396/794] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:39.279995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc41e-40be-4813-abd0-f35970a4a078 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:39.280859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc41e-40be-4813-abd0-f35970a4a078 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:39.281635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddc41e-40be-4813-abd0-f35970a4a078 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:39.284765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 397/795] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:39.291294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc41e-40be-4813-abd0-f35970a4a078 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:39.368016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:39.381014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:39.417039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ac9d-76eb-4182-a984-c2335cdf8366 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:39.433892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddc41e-40be-4813-abd0-f35970a4a078 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:39.464417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:39.465375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:39.465657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:39.465929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:39.477335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf0e9d-2d80-4a68-a5bc-50a2733aaf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:39.478597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf0e9d-2d80-4a68-a5bc-50a2733aaf88 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:39.479051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf0e9d-2d80-4a68-a5bc-50a2733aaf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:39.479317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf0e9d-2d80-4a68-a5bc-50a2733aaf88 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:39.479986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf0e9d-2d80-4a68-a5bc-50a2733aaf88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:39.480285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf0e9d-2d80-4a68-a5bc-50a2733aaf88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:39.480959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:39.481725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf0e9d-2d80-4a68-a5bc-50a2733aaf88 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:39.483173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:39.500703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 398/796] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:39.502160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4ba1c-af2a-4fcb-b651-1aecc3021079 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:39.502434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4ba1c-af2a-4fcb-b651-1aecc3021079 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:39.502703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4ba1c-af2a-4fcb-b651-1aecc3021079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:39.503018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:39.503917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:39.505154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:39.505154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:39.505154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:39.507654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 399/797] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:39.512782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:39.540460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4dc4ba1c-af2a-4fcb-b651-1aecc3021079 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:39.540460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4dc4ba1c-af2a-4fcb-b651-1aecc3021079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:39.647656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af676d27-41ca-477d-8120-d4e9e9581b74 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:39.678366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:39.693705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:39.694372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:39.694651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:39.694925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:39.708462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:39.708462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:39.709038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 399/798] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:39.710296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4ba1c-af2a-4fcb-b651-1aecc3021079 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:39.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:39.742843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:39.743870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:39.752471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 400/799] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:39.765643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:39.789477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:39.906202 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4ba1c-af2a-4fcb-b651-1aecc3021079 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:39.964134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:39.965807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:39.966807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:39.967297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:39.968534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:39.969057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:39.971765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:39.995815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 400/800] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:40.003030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:40.023522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6484e26-f9ed-4466-9803-8d038cedfb02 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.079090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.079713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.080115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.080616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.082121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:40.082644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.086527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.116293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 401/801] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 13 17:48:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:40.117936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:40.175824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:40.181151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c064895-cbb7-4e35-9371-1045c93ab9c5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.222782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.223849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.224536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.225032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.233845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.235820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:40.249567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:40.250553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:40.251469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:40.254134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 401/802] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:40.255337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:40.326112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:40.393426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bde788-5c4b-45a3-9b8b-ce23c265b208 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.437501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.438549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.439100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.439657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.440603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.442438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:40.453768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:40.454851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 17:48:40.455728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:40.463321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efcfb1-a4a5-4c8b-8819-900450685c90 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.463993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 402/803] 10.222.0.43 () {56 vars in 1129 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:40.465798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:40.531143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.532217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.532734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.533225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.555219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.558001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:40.586112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:40.587976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:40.598276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:40.598276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:40.598276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:40.598276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 402/804] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:40.598276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:40.619362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:40.620105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c7c0c-1e3b-4d1e-9dd9-b56aa164f147 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.632350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:40.669191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f81863-636c-48be-9127-6ac2e3d235e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.670887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81863-636c-48be-9127-6ac2e3d235e3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.671329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81863-636c-48be-9127-6ac2e3d235e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.671996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81863-636c-48be-9127-6ac2e3d235e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.673254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f81863-636c-48be-9127-6ac2e3d235e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:40.673736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f81863-636c-48be-9127-6ac2e3d235e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.676132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f81863-636c-48be-9127-6ac2e3d235e3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.695630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 403/805] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:40.697772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c253b-5058-423d-aebf-26bb5bfc976a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.698253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c253b-5058-423d-aebf-26bb5bfc976a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.698719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c253b-5058-423d-aebf-26bb5bfc976a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.701246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 404/806] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:40.702702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c253b-5058-423d-aebf-26bb5bfc976a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:40.756917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c8b5b-3959-4608-893d-eb9f993d90aa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.796225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416b641-93db-47ac-9a80-adfa8cb96242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.796888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416b641-93db-47ac-9a80-adfa8cb96242 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.797167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416b641-93db-47ac-9a80-adfa8cb96242 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.797432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416b641-93db-47ac-9a80-adfa8cb96242 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.798111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416b641-93db-47ac-9a80-adfa8cb96242 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:40.798413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416b641-93db-47ac-9a80-adfa8cb96242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.799912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416b641-93db-47ac-9a80-adfa8cb96242 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.805276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 403/807] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:40.806591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538a99-1119-41d9-ad1d-03a133b96437 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.806895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538a99-1119-41d9-ad1d-03a133b96437 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.807140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6538a99-1119-41d9-ad1d-03a133b96437 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.808839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 404/808] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:40.809451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6538a99-1119-41d9-ad1d-03a133b96437 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:40.869153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c253b-5058-423d-aebf-26bb5bfc976a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.903573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:40.909183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99572994-05b7-4d8b-873b-2eab375fadb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.909809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99572994-05b7-4d8b-873b-2eab375fadb7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.910082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99572994-05b7-4d8b-873b-2eab375fadb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.910347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99572994-05b7-4d8b-873b-2eab375fadb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.911007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99572994-05b7-4d8b-873b-2eab375fadb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:40.911302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99572994-05b7-4d8b-873b-2eab375fadb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.912803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99572994-05b7-4d8b-873b-2eab375fadb7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.925664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 405/809] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:40.927233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5095-1599-4824-a30c-d3880187f063 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.927568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5095-1599-4824-a30c-d3880187f063 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.927805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc5095-1599-4824-a30c-d3880187f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.944654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6538a99-1119-41d9-ad1d-03a133b96437 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:40.947227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0acc5095-1599-4824-a30c-d3880187f063 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:40.948391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0acc5095-1599-4824-a30c-d3880187f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:40.973652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:40.987789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:40.988759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:40.989170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:40.989541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:40.990638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=90f40fe8f20f4a5bbd201debda6a97a5) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:40.991033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:40.992648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:40.999418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 405/810] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/role_assignments?user.id=90f40fe8f20f4a5bbd201debda6a97a5&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:41.000278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:41.027805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 406/811] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:41.029695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a8fa0-b7c4-4a4a-9ee4-c2862eeb9b6d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:41.029975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a8fa0-b7c4-4a4a-9ee4-c2862eeb9b6d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:41.030241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a8fa0-b7c4-4a4a-9ee4-c2862eeb9b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:41.079081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e80a8fa0-b7c4-4a4a-9ee4-c2862eeb9b6d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:41.083302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e80a8fa0-b7c4-4a4a-9ee4-c2862eeb9b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:41.153477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75f37-69a1-4d2e-bedc-90cf1abe8262 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:41.210179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:41.211115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:41.211573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:41.212039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:41.213339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:41.213771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:41.216335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:41.226492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 406/812] 10.222.0.43 () {56 vars in 1124 bytes} [Tue Sep 13 17:48:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:41.227969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:41.234458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:41.252610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 407/813] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:41.254575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a8fa0-b7c4-4a4a-9ee4-c2862eeb9b6d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:41.415346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a719ecb-2094-4acc-a7f8-f03969c7eb08 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:41.428245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:41.453467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a8fa0-b7c4-4a4a-9ee4-c2862eeb9b6d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:41.480791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:41.481707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:41.482260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:41.482985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:41.496486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:41.499039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:41.520772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:41.522634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:41.531263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 407/814] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:41.538743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520fa67-4377-4867-b018-10328df116db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:41.540676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:41.555578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520fa67-4377-4867-b018-10328df116db None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:41.556082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520fa67-4377-4867-b018-10328df116db None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:41.556367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520fa67-4377-4867-b018-10328df116db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:41.557152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520fa67-4377-4867-b018-10328df116db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:41.558870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520fa67-4377-4867-b018-10328df116db None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:41.590269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 408/815] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:48:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:41.591451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520fa67-4377-4867-b018-10328df116db None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:41.673976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:41.825101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520fa67-4377-4867-b018-10328df116db None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:41.869802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:41.870470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:41.870744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:41.871012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] PATH_INFO: `/v3/services/network` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:41.871612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:41.873071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:41.883960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:41.885815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:41.887701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 409/816] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:41 2022] GET /identity/v3/services/network => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:41.890779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:41.898147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:41.951881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc129991-3d55-41b5-a079-c55a917fa793 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:42.024827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690dce7-09f5-47ad-b580-7632a6ae7573 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:42.040286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:42.041268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:42.041711 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:42.042134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:42.043543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:42.044014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:42.046996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:42.068854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:42.069872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:42.070355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:42.071013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:42.076693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 408/817] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:42.076693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:42.089908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:42.095170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:42.096165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:42.097277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:42.099838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 410/818] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:42.100902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:42.201563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:42.297815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89660c06-1850-4f84-82c5-adf86d94bce6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:42.315756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3a51-589c-41b5-abd8-f53aaed5caaf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:42.367618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:42.383466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:42.385368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:42.386123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:42.387198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/1ec763f1afac469196a5e897587813b8/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:42.390616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:42.391728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:42.392073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=1ec763f1afac469196a5e897587813b8, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:42.392233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:42.392698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:42.393870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] RBAC: Adding query filter params (name=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:42.394404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:42.396049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:42.397087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:42.407392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 409/819] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:48:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:42.414825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:42.428152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 411/820] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:42 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/1ec763f1afac469196a5e897587813b8/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:42.430277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:42.696587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6096d5-ad18-4854-9988-539f6541d2d9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:42.757170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:42.786439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:42.797470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:42.797737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:42.798090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:42.798716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:42.799027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:42.800515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:42.806800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 412/821] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:48:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:42.823108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:42.830503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c696c9-b0bf-4099-a51b-53ca0686d857 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:42.896017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:42.896838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:42.897114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:42.897404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:42.898169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:42.898475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:42.899901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:42.911317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 410/822] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:42.922496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:42.988472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:43.152749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385fe0c-3a0f-422d-adfb-63dd804a887c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:43.159933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f703a1-e661-40fd-ab50-3ce71db558d4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:43.274197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:43.275719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:43.276411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:43.277153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:43.278549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:43.280555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:43.281595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:43.282664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:43.283279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:43.283743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:43.308411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:43.311666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:43.336880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 413/823] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:48:42 2022] POST /identity/v3/endpoints => generated 337 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:43.352946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:43.354818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:43.357271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:43.357271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:43.357271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:43.364584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 411/824] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:43.370846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:43.455819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:43.577716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee7236-5479-471f-9861-56b2a186dbfc None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:43.666480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:43.667455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:43.667731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:43.668022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:43.668805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:43.669160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:43.671455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:43.696179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 414/825] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:43.703821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:43.904322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebb3b0-374d-4068-97d8-13ed412298cc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:43.907321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:43.977587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:43.978562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:43.978990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:43.979372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.003597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:44.005996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:44.029428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:44.030790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:44.032332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:44.036634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 412/826] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:48:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:44.040908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:44.209501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:44.216662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903909bc-6a49-4a50-a441-42af0b9b0bb4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:44.279748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69bd42-7886-4186-920b-ae130d4c1e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:44.280846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69bd42-7886-4186-920b-ae130d4c1e5e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:44.281327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69bd42-7886-4186-920b-ae130d4c1e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:44.281804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69bd42-7886-4186-920b-ae130d4c1e5e None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.283182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69bd42-7886-4186-920b-ae130d4c1e5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:44.283720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69bd42-7886-4186-920b-ae130d4c1e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:44.286561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69bd42-7886-4186-920b-ae130d4c1e5e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:44.322292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 415/827] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 13 17:48:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:44.327906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:44.517151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8f46b-365b-4f2c-a0ac-2164f8e85a20 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:44.543379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ff814-f973-485f-9290-d38e09e4b416 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:44.544078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ff814-f973-485f-9290-d38e09e4b416 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:44.544366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ff814-f973-485f-9290-d38e09e4b416 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.552967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 416/828] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:48:44.556986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307eee-c81d-451a-94bc-98550fbff12a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:44.557391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307eee-c81d-451a-94bc-98550fbff12a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:44.557861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307eee-c81d-451a-94bc-98550fbff12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.584139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:44.585292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:44.585773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:44.586338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.589008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-90307eee-c81d-451a-94bc-98550fbff12a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:44.591072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-90307eee-c81d-451a-94bc-98550fbff12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:44.607628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:44.610491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:44.640099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:44.642433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:44.644617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:44.644617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:44.644617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:44.648852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 413/829] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:44.652873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:44.700876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:44.724881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 417/830] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:44.812738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2fc9-61e1-4346-83fe-14845f1ecd85 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:44.813407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2fc9-61e1-4346-83fe-14845f1ecd85 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:44.813898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa2fc9-61e1-4346-83fe-14845f1ecd85 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.815765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 418/831] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:44.821736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9d945-7c66-4b07-9360-cef41e76eff1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:44.822346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9d945-7c66-4b07-9360-cef41e76eff1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:44.822814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9d945-7c66-4b07-9360-cef41e76eff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.841995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-75d9d945-7c66-4b07-9360-cef41e76eff1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:44.843588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75d9d945-7c66-4b07-9360-cef41e76eff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:44.845490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae282e-cb81-46ab-a2a3-80b7bae65e89 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:44.886416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:44.887401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:44.887677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:44.887946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:44.888789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:44.889094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:44.891103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:44.909632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 414/832] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:44.910864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:44.928528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 419/833] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:44.992983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:45.021454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d47c10-583e-46aa-a20f-7960213f86c2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:45.058434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:45.059078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.059357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.059626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.067499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:45.069030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:45.081555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:45.082528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:45.083409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:45.087622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 415/834] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:45.089202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9d945-7c66-4b07-9360-cef41e76eff1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:45.120904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:45.154501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:45.248483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9d945-7c66-4b07-9360-cef41e76eff1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:45.300005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:45.301622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.302353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.303025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.304716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:45.305908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:45.309004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:45.314802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec14b6-d6dd-413a-ad97-69e4ee53d6af None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:45.331025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 420/835] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:45.331832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:45.377384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:45.378070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.378344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.378616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.379252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:45.381011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:45.383854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:45.395353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:45.397152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:45.399761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:45.411341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 416/836] 10.222.0.43 () {56 vars in 1132 bytes} [Tue Sep 13 17:48:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:45.412725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:45.496660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:45.526368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85446fcf-88eb-4adc-b833-a29ef221cab2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:45.576100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:45.577171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.577512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.577898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.578589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:45.580080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:45.580973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739be71-fd7a-4e83-918f-499d85a8ba4c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:45.595236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:45.598155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:45.602483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 421/837] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:45.605345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:45.631260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c965368-6da1-4547-a391-d3dd69d3f150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:45.632060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c965368-6da1-4547-a391-d3dd69d3f150 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.632453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c965368-6da1-4547-a391-d3dd69d3f150 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.632815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c965368-6da1-4547-a391-d3dd69d3f150 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/1e6a922a2aea4d3ea1feadbd25cb32cf/roles/cb2c16a6113142179869b36ff210c590` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.650245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c965368-6da1-4547-a391-d3dd69d3f150 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=1e6a922a2aea4d3ea1feadbd25cb32cf, role_id=cb2c16a6113142179869b36ff210c590)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:45.651865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c965368-6da1-4547-a391-d3dd69d3f150 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:45.667020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 417/838] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:48:45 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/1e6a922a2aea4d3ea1feadbd25cb32cf/roles/cb2c16a6113142179869b36ff210c590 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:48:45.668740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79f500-c87b-4fe6-b0b5-102f07c30bb4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.669131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79f500-c87b-4fe6-b0b5-102f07c30bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.669555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79f500-c87b-4fe6-b0b5-102f07c30bb4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.671083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 418/839] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:45.672452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79f500-c87b-4fe6-b0b5-102f07c30bb4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:45.760835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd43d6-8842-428c-a5b6-6f1f3b4b55e1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:45.770154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:45.812450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f115a-7265-42af-ba05-3f8909f207d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:45.813378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f115a-7265-42af-ba05-3f8909f207d3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.813817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f115a-7265-42af-ba05-3f8909f207d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.814257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f115a-7265-42af-ba05-3f8909f207d3 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.815137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f115a-7265-42af-ba05-3f8909f207d3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:45.815558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f115a-7265-42af-ba05-3f8909f207d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:45.817202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f115a-7265-42af-ba05-3f8909f207d3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:45.828523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 422/840] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:48:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:45.828523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32329cd8-27b0-4bbb-b427-56e11081b4c2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.828523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32329cd8-27b0-4bbb-b427-56e11081b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.828523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32329cd8-27b0-4bbb-b427-56e11081b4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.846677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:45.856511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-32329cd8-27b0-4bbb-b427-56e11081b4c2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:45.858730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-32329cd8-27b0-4bbb-b427-56e11081b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:45.881517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79f500-c87b-4fe6-b0b5-102f07c30bb4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:45.936792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:45.937879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:45.937879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:45.938150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:45.938558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:45.938859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:45.940298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:45.945940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 419/841] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:45.947390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:45.976937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 423/842] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:45.976937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32329cd8-27b0-4bbb-b427-56e11081b4c2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:46.081026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9548e5-e7ad-4c4d-805c-ad957e4bcbea None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:46.100923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:46.142626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:46.143741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:46.144049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:46.144478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:46.145614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:46.146335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:46.147921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:46.154946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 420/843] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:48:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:46.186725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32329cd8-27b0-4bbb-b427-56e11081b4c2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:46.247087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:46.256950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:46.258337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:46.258826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:46.259319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:46.260750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:46.261289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:46.264870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:46.269274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:46.284068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 424/844] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:46.426982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3a0a3-cbcd-4922-b106-25b66a2dc304 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:46.470940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:46.471600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:46.471894 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:46.472146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:46.485406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:46.490496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:46.491482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:46.492566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 421/845] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:46.500681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:46.620298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cff69-85b3-4b28-977d-2ece0849576a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:46.658098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:46.658733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:46.659017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:46.659286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:46.659978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:46.660283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:46.661701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:46.672190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 425/846] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:48:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:46.678595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:46.733586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:46.811961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0443-fbdf-40b5-a318-2acdd0f61b7d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:46.856409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:46.857137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:46.857413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:46.857766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:46.869958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:46.871453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:46.888610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:46.889633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:46.890736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:46.890736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:46.890736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:46.890736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 422/847] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:46.897789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:46.966277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:47.037686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415d413-476b-4055-8f27-ab3ce5462e62 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:47.099964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:47.101446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:47.102163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:47.102805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:47.104243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:47.105045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:47.108046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:47.136371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 426/848] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:47.141654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:47.295564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:47.320174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1b15c-f38a-4c73-9cb9-856dcf9c6b6f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:47.381589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:47.382255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:47.382529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:47.382796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:47.392074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:47.393587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:47.406077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:47.407097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:47.408048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 423/849] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:48:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:47.417108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:47.526498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca20f14-da0e-405d-83b4-212ec319d0e2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:47.557745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:47.563966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:47.564634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:47.564914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:47.565184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:47.565942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:47.566655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:47.568536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:47.588696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 427/850] 10.222.0.43 () {56 vars in 1135 bytes} [Tue Sep 13 17:48:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:47.597935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:47.780086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ca30d-507b-4cfd-84d1-55f5eefdf70e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:47.838797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:47.839341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:47.839628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:47.839895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:47.859819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:47.915929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:47.917444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:48.028253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 424/851] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:48:48.037083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:48.148696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49762c26-99c6-4768-b183-0164831fd03c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:48.185914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:48.186566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.186844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.187117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.198680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:48.200163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:48.216854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:48.217906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:48.218805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:48.218805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:48.218805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:48.218805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 428/852] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:48.224642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:48.278652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:48.335911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d46d2-bc6c-4d91-abaa-e805fd99675f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:48.374465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9cc98-cfb9-464d-a2dd-3a41dcaee112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:48.375459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9cc98-cfb9-464d-a2dd-3a41dcaee112 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.375940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9cc98-cfb9-464d-a2dd-3a41dcaee112 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.376450 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9cc98-cfb9-464d-a2dd-3a41dcaee112 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.377336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9cc98-cfb9-464d-a2dd-3a41dcaee112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:48.377840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9cc98-cfb9-464d-a2dd-3a41dcaee112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:48.379468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9cc98-cfb9-464d-a2dd-3a41dcaee112 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:48.393898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 425/853] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:48.404714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:48.445774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:48.546605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d5a42-3bf8-4f8d-bb08-2bf432cdb747 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:48.587430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:48.588110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.588391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.588711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.597081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:48.598595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:48.599213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6769-4513-449f-8abe-c0fa4ad0cddb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.600199 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6769-4513-449f-8abe-c0fa4ad0cddb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.601008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6769-4513-449f-8abe-c0fa4ad0cddb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.610802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 426/854] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:48.611784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:48.613919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:48.615733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 429/855] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:48.623825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54c75e-ca61-42eb-8423-8f2c9c47f3d1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.624491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:48.624849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54c75e-ca61-42eb-8423-8f2c9c47f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.625561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54c75e-ca61-42eb-8423-8f2c9c47f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.662061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0c54c75e-ca61-42eb-8423-8f2c9c47f3d1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:48.664882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0c54c75e-ca61-42eb-8423-8f2c9c47f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:48.740623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:48.752561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8790137-cc21-4649-ae0a-943c21c073aa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:48.793378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:48.794202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.794590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.794963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.795817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:48.796248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:48.798293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:48.819895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 430/856] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:48.820173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 427/857] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:48.826567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:48.873953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eb7f6-ba04-4c33-89de-fe2ba21b9413 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.874307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eb7f6-ba04-4c33-89de-fe2ba21b9413 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.874501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eb7f6-ba04-4c33-89de-fe2ba21b9413 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.876169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 428/858] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:48:48.882017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8d60a-3460-4cf1-bd67-e5aa1c07e69f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:48.882346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8d60a-3460-4cf1-bd67-e5aa1c07e69f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:48.882646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8d60a-3460-4cf1-bd67-e5aa1c07e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:48.903143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-16f8d60a-3460-4cf1-bd67-e5aa1c07e69f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:48.904465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-16f8d60a-3460-4cf1-bd67-e5aa1c07e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:48.950170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:49.025030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 429/859] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:49.034070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58867f86-df75-4ecc-9c9a-93a29338e698 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:49.078135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3174ad-2cfe-4178-a622-a0fcd671c7ac None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:49.078421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3174ad-2cfe-4178-a622-a0fcd671c7ac None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:49.078769 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3174ad-2cfe-4178-a622-a0fcd671c7ac None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:49.081092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 430/860] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:48:49.086489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de220-8006-407d-8f17-796b0b287148 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:49.086769 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de220-8006-407d-8f17-796b0b287148 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:49.087083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de220-8006-407d-8f17-796b0b287148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:49.112036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f90de220-8006-407d-8f17-796b0b287148 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:49.114090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f90de220-8006-407d-8f17-796b0b287148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:49.116887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:49.118364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:49.118628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:49.118910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:49.119685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1ec763f1afac469196a5e897587813b8) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:49.120001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:49.121551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:49.129842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 431/861] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:48 2022] GET /identity/v3/role_assignments?user.id=1ec763f1afac469196a5e897587813b8&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:49.130652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:49.155013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:49.231423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 431/862] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:49.343006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11690871-fd39-4ec1-971f-cd75ecb6f377 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:49.388339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:49.389038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:49.389308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:49.389587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:49.402365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:49.407983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:49.410086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:49.411849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 432/863] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:48:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:49.415827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de220-8006-407d-8f17-796b0b287148 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:49.496903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:49.564831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:49.575379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de220-8006-407d-8f17-796b0b287148 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:49.596968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:49.636140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040e068-cda7-4d6e-a3ee-9a7d4b916a73 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:49.642624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:49.643645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:49.644075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:49.644524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:49.645494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:49.646070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:49.649089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:49.664729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 432/864] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:48:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:49.666253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:49.694267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:49.695663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:49.696147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:49.696644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:49.697529 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:49.699118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:49.724641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 433/865] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:48:49 2022] POST /identity/v3/services => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:49.724641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:49.825478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df284d4f-cb35-4ac7-a006-cd37aa64932f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:49.884376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:49.885086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:49.886166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:49.886867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:49.887615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:49.889159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:49.915339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 433/866] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:48:49 2022] POST /identity/v3/services => generated 269 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:49.942890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd0782-79e1-49a2-a9f4-61f5e8410d9d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:50.013107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:50.023298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:50.024411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:50.024937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:50.025340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:50.041837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:50.043861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:50.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:50.065901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:50.067739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:50.067739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:50.067739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:50.067739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 434/867] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:50.074053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:50.097484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:50.226461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2adafc-5eac-43e2-85cb-f94a9c9b19b8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:50.268279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:50.269331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:50.269331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:50.269578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:50.270110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:50.270411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:50.271895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:50.287241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 434/868] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:50.293000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:50.413700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445d989-d9d2-4438-a3ef-6a8750efd151 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:50.452937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:50.453607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:50.453889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:50.454163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:50.463571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:50.465126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:50.478327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:50.481483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:50.488629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 435/869] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:48:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:50.490675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:50.586100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:50.616047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885cec3-a3eb-4dad-b67f-cb926dcd4604 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:50.674811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:50.675683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:50.676104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:50.676546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:50.677887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:50.678449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:50.681187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:50.716078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 435/870] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 13 17:48:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:50.723048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:50.774762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:50.832708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e7c71-e453-4048-99ea-2956fba9c3db None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:50.867885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:50.868586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:50.868876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:50.869121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:50.946454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:50.947904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:51.001057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:51.051412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 436/871] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:50 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:48:51.057837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:51.235313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd520288-9355-4ba2-b474-882e39df0927 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:51.275660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:51.276712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:51.276712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:51.277025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:51.279188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:51.289434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:51.290941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:51.307596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:51.309446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:51.311174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:51.311174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:51.311174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:51.311174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 436/872] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:51.314591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:51.420629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478f80-da7b-426e-8ae1-7468ee3906c9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:51.455785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:51.456386 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:51.456703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:51.456975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:51.457640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:51.457931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:51.459400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:51.471684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 437/873] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:51.477552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:51.618756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d2c0-5e10-4646-9014-f8fd45581d4f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:51.679972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:51.681478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:51.682206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:51.682900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:51.696049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:51.699125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:51.720750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:51.722498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:51.724173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:51.728335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 437/874] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:51.734013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:51.739205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:51.844120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac217c84-488c-4175-97b2-396e10c69451 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:51.884298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42826a-eb76-48bd-9a72-95a19e9b05c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:51.885128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42826a-eb76-48bd-9a72-95a19e9b05c1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:51.885536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42826a-eb76-48bd-9a72-95a19e9b05c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:51.885923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42826a-eb76-48bd-9a72-95a19e9b05c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:51.887332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42826a-eb76-48bd-9a72-95a19e9b05c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:51.887782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42826a-eb76-48bd-9a72-95a19e9b05c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:51.890033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42826a-eb76-48bd-9a72-95a19e9b05c1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:51.909235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 438/875] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:51.916164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:51.988014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:52.051315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb703b0-bb3a-4bfe-8e38-26fe2e813e52 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:52.115951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8af29d-2c62-4c9e-a678-679d7455fecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:52.117156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8af29d-2c62-4c9e-a678-679d7455fecb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:52.117641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8af29d-2c62-4c9e-a678-679d7455fecb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:52.118111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8af29d-2c62-4c9e-a678-679d7455fecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:52.119558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8af29d-2c62-4c9e-a678-679d7455fecb None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1e6a922a2aea4d3ea1feadbd25cb32cf) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:52.120082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8af29d-2c62-4c9e-a678-679d7455fecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:52.123031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8af29d-2c62-4c9e-a678-679d7455fecb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:52.136319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 438/876] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:51 2022] GET /identity/v3/role_assignments?user.id=1e6a922a2aea4d3ea1feadbd25cb32cf&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:48:52.772512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4f851-f6f8-4de0-b1fc-9e07dbe3b25e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:52.773069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4f851-f6f8-4de0-b1fc-9e07dbe3b25e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:52.773601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4f851-f6f8-4de0-b1fc-9e07dbe3b25e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:52.776394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 439/877] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:52.782295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cead19-bde9-42ef-a193-845330eb595e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:52.782857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cead19-bde9-42ef-a193-845330eb595e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:52.782857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cead19-bde9-42ef-a193-845330eb595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:52.804032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-40cead19-bde9-42ef-a193-845330eb595e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:52.805346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-40cead19-bde9-42ef-a193-845330eb595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:52.817115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220e08-e176-46a7-ad45-7e892033a890 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:52.817733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220e08-e176-46a7-ad45-7e892033a890 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:52.818239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220e08-e176-46a7-ad45-7e892033a890 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:52.820096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 440/878] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:48:52.827090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2cb9-705e-4c4b-a9ec-1bbc6cc286d0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:52.827664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2cb9-705e-4c4b-a9ec-1bbc6cc286d0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:52.828151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2cb9-705e-4c4b-a9ec-1bbc6cc286d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:52.849389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f36c2cb9-705e-4c4b-a9ec-1bbc6cc286d0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:52.850720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f36c2cb9-705e-4c4b-a9ec-1bbc6cc286d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:52.886287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 439/879] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:52.944867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:52.958450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 441/880] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:52.982621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22b37d-8acf-4c8f-bf1e-4fc8c0fa76c5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:52.983098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22b37d-8acf-4c8f-bf1e-4fc8c0fa76c5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:52.983578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22b37d-8acf-4c8f-bf1e-4fc8c0fa76c5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:52.986738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 440/881] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:52.995867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b953ef3-7eb6-491b-b661-d98db2520cdd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:52.996353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b953ef3-7eb6-491b-b661-d98db2520cdd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:52.996927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b953ef3-7eb6-491b-b661-d98db2520cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:53.028544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0b953ef3-7eb6-491b-b661-d98db2520cdd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:53.030534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0b953ef3-7eb6-491b-b661-d98db2520cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:53.151349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 442/882] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:53.309307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22b37d-8acf-4c8f-bf1e-4fc8c0fa76c5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:53.367347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:53.382791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b953ef3-7eb6-491b-b661-d98db2520cdd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:53.441693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22b37d-8acf-4c8f-bf1e-4fc8c0fa76c5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:53.514736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:53.516720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:53.517133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:53.517610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:53.519047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:53.522024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:53.542091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:53.544033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:53.545917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 441/883] 10.222.0.43 () {56 vars in 1132 bytes} [Tue Sep 13 17:48:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:53.550164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:53.582119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b953ef3-7eb6-491b-b661-d98db2520cdd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:53.609501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:53.624712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:53.625365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:53.625638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:53.625911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:53.626512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:53.627911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:53.638355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:53.639455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:53.640285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 443/884] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:48:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:48:53.641121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:53.673869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dde502-1049-40b8-9cd1-60a51d404955 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:53.677265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:53.729318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:53.730420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:53.730793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:53.731168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:53.732130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:53.732581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:53.735581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:53.745346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 442/885] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:48:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:53.745346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:53.780742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e84138-cf54-4a29-ac5e-22a66929a793 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:53.824952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:53.825602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:53.825871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:53.826138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:53.826747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:53.828130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:53.837897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:53.839591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:53.842328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 444/886] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:53.844101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:53.875403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dcf71-92fa-41a6-912d-787ef0a79916 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:53.918595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:53.919218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:53.919490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:53.919766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:53.920411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:53.920740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:53.920911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:53.922147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:53.927492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 443/887] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:48:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:53.928354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:53.962994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:54.003434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b48c64-2649-4626-96c4-b4eec74783bb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.047799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.048956 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.049365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.049774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.050619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:54.051046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.052918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.065993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 445/888] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:48:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:54.066210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5218cbe-a347-4bfd-ac96-cab02b699448 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.066293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:54.109925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.110448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.110723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.110989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.111655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:54.111954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.113475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.126783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 444/889] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:54.126783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:54.225344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168914e-3a2e-458c-a331-4a428757f61d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.246446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:54.274174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.274868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.275317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.275763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.276892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:54.277472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.279813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.292880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 446/890] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:48:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:54.293417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:54.294323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:54.302798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd5da2-4b94-47ee-966d-3a286d0a184f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.373262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.374323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.374814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.375446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.377189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] RBAC: Adding query filter params (interface=public, service_id=1f03c53eee2941758faf8e0b5e6337e2, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:54.377759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.381042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.393177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 445/891] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:48:54 2022] GET /identity/v3/endpoints?service_id=1f03c53eee2941758faf8e0b5e6337e2&interface=public®ion_id=RegionOne => generated 194 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:54.402271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:54.429147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfc5d1-dddf-4f1a-8a25-0b6d610f1982 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.490736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.491846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.492339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.492875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.494143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:54.494664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.497507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.507669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 447/892] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:54.518329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:54.607569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c992920-1715-4f63-bf6c-97bac4722dec None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.617958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:54.660373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.662336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.663118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.663651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.664511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.666150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.677035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 446/893] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:48:54 2022] GET /identity/v3/services => generated 1268 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:48:54.719174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662503fd-5b19-40a6-9305-90e5cde9f074 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.759415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f847c-b630-490c-8491-c431297588df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.760129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f847c-b630-490c-8491-c431297588df None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.760444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f847c-b630-490c-8491-c431297588df None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.760715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f847c-b630-490c-8491-c431297588df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.761383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f847c-b630-490c-8491-c431297588df None admin] RBAC: Adding query filter params (interface=public, service_id=cd1f3be8f20f4b809fed2b47a52f6547, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:54.762054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f847c-b630-490c-8491-c431297588df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.763468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f847c-b630-490c-8491-c431297588df None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.776888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 448/894] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:48:54 2022] GET /identity/v3/endpoints?service_id=cd1f3be8f20f4b809fed2b47a52f6547&interface=public®ion_id=RegionOne => generated 194 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:54.779519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:54.781496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef37af-0512-49fc-858d-acfe489e166d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.782031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef37af-0512-49fc-858d-acfe489e166d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.782031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef37af-0512-49fc-858d-acfe489e166d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.783752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 449/895] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:48:54.790696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186530-cf2b-4788-8008-16d79437c85f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.791041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186530-cf2b-4788-8008-16d79437c85f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.791232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186530-cf2b-4788-8008-16d79437c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.814210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7e186530-cf2b-4788-8008-16d79437c85f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:54.815732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e186530-cf2b-4788-8008-16d79437c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:54.816927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:54.908695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 450/896] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:54.926045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325118f-e648-481f-b835-4856ec8e3463 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:54.967639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:54.968833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:54.969379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:54.969853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:54.970713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:54.972403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:54.979962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 447/897] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:48:54 2022] GET /identity/v3/services => generated 1268 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:48:55.255440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186530-cf2b-4788-8008-16d79437c85f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:55.332129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:55.427773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186530-cf2b-4788-8008-16d79437c85f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:55.469659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:55.470725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:55.471213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:55.471691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:55.484247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:48:55.489788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:55.490939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:48:55.491850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:55.494937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 451/898] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 13 17:48:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:55.504931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:55.677076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:55.700399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff433b-806c-4bc8-a87f-534033007ceb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:55.768967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:55.770383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:55.771072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:55.771936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:55.773341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:55.774090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:55.776770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:55.791773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 448/899] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 13 17:48:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:55.800705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:55.970022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:55.976334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2d315-3ed2-405c-b27f-b6c4ad78e373 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:56.039785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:56.040899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:56.041380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:56.041846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:56.061399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:56.064410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:56.093889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:56.095683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:56.097418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:56.097418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:56.097418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:56.097418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 452/900] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:56.105454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:56.251888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c62e6-496f-420b-8c14-0e6fdacbaa81 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:56.300453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:56.300453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:56.300453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:56.300453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:56.301163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:56.301163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:56.301163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:56.320577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 449/901] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:56.332363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:56.510469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce00cb-a904-4794-a57d-28e7f76be541 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:56.570827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:56.571626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:56.572059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:56.572363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:56.584122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:56.586206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:48:56.604842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:56.606545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:56.608659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 453/902] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:48:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:56.620040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:56.642731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:56.824291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba8a8a-c25d-4bf2-ae1c-38f77d39b094 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:56.892942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:56.894520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:56.895281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:56.896055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:56.897875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:56.898717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:56.901649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:56.943124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 450/903] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 13 17:48:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:56.949688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:56.975867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:57.138312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15286b37-017e-4800-872b-d0c0e082af26 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:57.200276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2d013-80e7-4d40-88ce-ff33dff80589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:57.201243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d013-80e7-4d40-88ce-ff33dff80589 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.201598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d013-80e7-4d40-88ce-ff33dff80589 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.202052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2d013-80e7-4d40-88ce-ff33dff80589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.327393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2d013-80e7-4d40-88ce-ff33dff80589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:57.328925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2d013-80e7-4d40-88ce-ff33dff80589 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:57.342133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:57.443237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 454/904] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:48:56 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:48:57.453855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:57.459350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50dc9a-d08e-4a8c-9512-51df39278bf2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.459641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50dc9a-d08e-4a8c-9512-51df39278bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.460018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50dc9a-d08e-4a8c-9512-51df39278bf2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.461763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 455/905] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:48:57.468344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab32f7-43e9-4a29-862e-92dbdadb87ab None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.468706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab32f7-43e9-4a29-862e-92dbdadb87ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.469033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab32f7-43e9-4a29-862e-92dbdadb87ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.489589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-27ab32f7-43e9-4a29-862e-92dbdadb87ab None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:57.491014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-27ab32f7-43e9-4a29-862e-92dbdadb87ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:57.572378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 456/906] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:57.574678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec68c2-e452-4397-b98c-a862734c3e10 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.575016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec68c2-e452-4397-b98c-a862734c3e10 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.575295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec68c2-e452-4397-b98c-a862734c3e10 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.576954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 457/907] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:57.584981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a83e-ac26-443d-945a-583264ceb757 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.585283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a83e-ac26-443d-945a-583264ceb757 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.585617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9a83e-ac26-443d-945a-583264ceb757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.599487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:57.607236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3fc9a83e-ac26-443d-945a-583264ceb757 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:57.608649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3fc9a83e-ac26-443d-945a-583264ceb757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:57.653653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:57.664729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c152-0791-418b-a068-25ca43122bbf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:57.700949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 458/908] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:48:57.702372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0b1b1-1060-4751-9340-c6870bb9012f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.702718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0b1b1-1060-4751-9340-c6870bb9012f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.703059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0b1b1-1060-4751-9340-c6870bb9012f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.704719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 459/909] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:48:57.713027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153abed-e50f-40bb-b8b0-a7771bd63259 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.713482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153abed-e50f-40bb-b8b0-a7771bd63259 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.713826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153abed-e50f-40bb-b8b0-a7771bd63259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.717793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:57.718766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:57.719190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:57.719599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:57.735623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8153abed-e50f-40bb-b8b0-a7771bd63259 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:48:57.737063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8153abed-e50f-40bb-b8b0-a7771bd63259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:48:57.741572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:57.743434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:48:57.765365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:57.766470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:48:57.767373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:48:57.767373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:48:57.767373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:57.770213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 451/910] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:57.780346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:57.836246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 460/911] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:48:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:48:57.929270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153abed-e50f-40bb-b8b0-a7771bd63259 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:57.977379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f96ae3-e1ac-4f55-8c3b-0b78904e23d2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.029385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:58.053605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:58.055359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.056517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.056957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.057327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.058108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:58.058511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.059463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153abed-e50f-40bb-b8b0-a7771bd63259 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.060319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.079607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 452/912] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:58.081978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.106082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.107213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.107601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.107976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.110988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.113928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.139952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 461/913] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:48:57 2022] POST /identity/v3/services => generated 241 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:58.140926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.224225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf003a6-9373-4466-b35d-c5b5dc3f4567 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.274254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.275261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.275748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.276228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.289802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.292649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:48:58.315191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:58.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 453/914] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:58.325069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.330507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91232a-afd8-4552-aebc-522fc3ea2f4e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.380535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:58.385213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.386210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.387054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.388321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.388678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.391227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:58.410196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:58.412465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:48:58.414656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 462/915] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:58.418783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.454363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:58.492061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b45bcf-7479-4acd-a663-6095f4a7d7b2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.551285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.552301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.552831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.553242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.554176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.556627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.556861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff8ac-8b9e-4045-8edd-37f2fcf42bdd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:48:58.571870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:48:58.574007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:48:58.575821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 454/916] 10.222.0.43 () {56 vars in 1132 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:48:58.579124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.616679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.617627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.618073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.618519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.619715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:58.620295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.622868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.643600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 463/917] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:58.644396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.726236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:58.731373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c598c4-3384-48e4-91e8-4bb33ca71d68 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.775642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:58.778110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.779194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.779645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.780474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.781283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:58.781898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.784455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.794707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf68773-2d90-424a-8356-1ddf7817be46 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.802033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 455/918] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:58.803933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.834368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.835064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.835271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.835538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.836262 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:58.836516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.837888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:58.848461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 464/919] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:58.848461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:58.955061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754639e-0314-44a6-aa78-1ae46dd9b8c4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.973331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4db88-e39d-4199-81d3-04fbb468b99f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:58.996158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:58.996788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:58.997066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:58.997333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:58.998080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] RBAC: Adding query filter params (role.id=b4532c8703f94b348ec0e808e4543d64, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1e6a922a2aea4d3ea1feadbd25cb32cf) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:58.998366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:58.999833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:59.006219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 456/920] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/role_assignments?user.id=1e6a922a2aea4d3ea1feadbd25cb32cf&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=b4532c8703f94b348ec0e808e4543d64 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:59.007164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:59.039537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:59.041544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:59.042687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:59.044905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:59.046371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:59.049110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:59.050626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:59.057560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:59.072485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 465/921] 10.222.0.43 () {56 vars in 1114 bytes} [Tue Sep 13 17:48:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:59.082010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:59.117225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:59.205689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145f1a6-2401-4808-acd8-f884f0844f3b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:59.229650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7daec-cff7-4001-9c42-b39c213f6710 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:59.274367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c2111-ff35-4b26-aca1-927babcb94fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:59.275424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c2111-ff35-4b26-aca1-927babcb94fa None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:59.275913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c2111-ff35-4b26-aca1-927babcb94fa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:59.276378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c2111-ff35-4b26-aca1-927babcb94fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:59.277460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c2111-ff35-4b26-aca1-927babcb94fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:59.278055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:59.279165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:59.279597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c2111-ff35-4b26-aca1-927babcb94fa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:59.279703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:59.280161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:59.281341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:48:59.281865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:59.285024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:59.305144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 457/922] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:48:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:48:59.307070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:48:59.309186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 466/923] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:48:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:48:59.501708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:48:59.522886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379b3d6-8a45-44bf-87a1-f83bd8b4dbe7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:48:59.571539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e98c2-93de-497f-a951-7d1953933de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:48:59.572642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e98c2-93de-497f-a951-7d1953933de2 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:48:59.573115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e98c2-93de-497f-a951-7d1953933de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:48:59.573576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e98c2-93de-497f-a951-7d1953933de2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:48:59.574504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e98c2-93de-497f-a951-7d1953933de2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:48:59.576093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e98c2-93de-497f-a951-7d1953933de2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:48:59.603545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 458/924] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:48:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:01.253590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13eeaf-ff24-43a3-8fc1-cc451ba031cb None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:01.254144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13eeaf-ff24-43a3-8fc1-cc451ba031cb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:01.254144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13eeaf-ff24-43a3-8fc1-cc451ba031cb None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:01.255975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 467/925] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:01.262553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19148c-aa94-47a6-9b3f-5f0c24817501 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:01.263082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19148c-aa94-47a6-9b3f-5f0c24817501 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:01.263082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19148c-aa94-47a6-9b3f-5f0c24817501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:01.283311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8c19148c-aa94-47a6-9b3f-5f0c24817501 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:01.284581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8c19148c-aa94-47a6-9b3f-5f0c24817501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:01.417854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 459/926] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:01.576047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9486e9-2d61-49de-9211-439eb27a0a23 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:01.576325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9486e9-2d61-49de-9211-439eb27a0a23 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:01.576626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9486e9-2d61-49de-9211-439eb27a0a23 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:01.578226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 468/927] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:01.584230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec979941-bbd7-48a2-8936-a3eec43eeafe None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:01.584552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec979941-bbd7-48a2-8936-a3eec43eeafe None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:01.584813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec979941-bbd7-48a2-8936-a3eec43eeafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:01.604592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ec979941-bbd7-48a2-8936-a3eec43eeafe None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:01.606139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ec979941-bbd7-48a2-8936-a3eec43eeafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:01.674287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:01.679379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 460/928] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:01.694850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9486e9-2d61-49de-9211-439eb27a0a23 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:01.775828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:01.838773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9486e9-2d61-49de-9211-439eb27a0a23 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:01.902078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87e784-2686-452d-b581-ddab57ed5d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:01.903544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87e784-2686-452d-b581-ddab57ed5d33 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:01.904325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87e784-2686-452d-b581-ddab57ed5d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:01.905079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87e784-2686-452d-b581-ddab57ed5d33 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:01.925351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87e784-2686-452d-b581-ddab57ed5d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5e87e784-2686-452d-b581-ddab57ed5d33 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:01.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:01.937785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:01.939313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:01.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 469/929] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 13 17:49:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:01.946010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272e19c-6b4d-413f-bde8-ab7d48d1f78f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:01.946010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272e19c-6b4d-413f-bde8-ab7d48d1f78f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:01.946010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272e19c-6b4d-413f-bde8-ab7d48d1f78f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:01.946010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 461/930] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:01.949490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272e19c-6b4d-413f-bde8-ab7d48d1f78f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:01.949839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b293d-54ec-4560-845c-22f8e5de153c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:01.950229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b293d-54ec-4560-845c-22f8e5de153c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:01.950485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b293d-54ec-4560-845c-22f8e5de153c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:01.973119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c46b293d-54ec-4560-845c-22f8e5de153c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:01.974471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c46b293d-54ec-4560-845c-22f8e5de153c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:02.045992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 470/931] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:02.056726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b293d-54ec-4560-845c-22f8e5de153c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.093150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272e19c-6b4d-413f-bde8-ab7d48d1f78f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.131495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.132151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.132441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.132739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.133341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.134762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:02.146973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:02.148052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:02.148958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 462/932] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:49:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:02.153969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:02.156095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.202025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b293d-54ec-4560-845c-22f8e5de153c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.248510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:02.250051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.250683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.250959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.251229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.251912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:02.252215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.253746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.264940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 471/933] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:02.273911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.299439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdb602-94da-4463-841d-5cbaf3b524a9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.338647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.339282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.339559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.339829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.340508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:02.340814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.342229 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.348022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 463/934] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:02.348819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.390593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfabca-4f69-4f9c-838f-8652750caa61 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.413240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:02.429195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87788a9-9069-45fe-9cce-53ab33a93ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.429848 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87788a9-9069-45fe-9cce-53ab33a93ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.430129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87788a9-9069-45fe-9cce-53ab33a93ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.430428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87788a9-9069-45fe-9cce-53ab33a93ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.442090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87788a9-9069-45fe-9cce-53ab33a93ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.443624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87788a9-9069-45fe-9cce-53ab33a93ef4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b87788a9-9069-45fe-9cce-53ab33a93ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:02.462429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:02.463692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:02.464787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:02.464787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:02.464787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:02.464787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 472/935] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:02.465060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736cbb-0c1d-480b-b9f9-fc2574ed917b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.465229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736cbb-0c1d-480b-b9f9-fc2574ed917b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.465595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736cbb-0c1d-480b-b9f9-fc2574ed917b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.471487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 473/936] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:02.471487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736cbb-0c1d-480b-b9f9-fc2574ed917b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.490389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8d5f4-f9ea-44a9-ab5f-f954cdc75033 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.520823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:02.529858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57e9ce-9865-4ce1-878b-cbf9a62bcae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.530505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57e9ce-9865-4ce1-878b-cbf9a62bcae0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.530780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57e9ce-9865-4ce1-878b-cbf9a62bcae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.531052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57e9ce-9865-4ce1-878b-cbf9a62bcae0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.531714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57e9ce-9865-4ce1-878b-cbf9a62bcae0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.533161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57e9ce-9865-4ce1-878b-cbf9a62bcae0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-eb57e9ce-9865-4ce1-878b-cbf9a62bcae0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:02.542603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:02.543854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:02.545240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 464/937] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:02.546308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf88fbe-f20d-4c20-b3ba-3aa27a33322f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.546308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf88fbe-f20d-4c20-b3ba-3aa27a33322f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.546308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf88fbe-f20d-4c20-b3ba-3aa27a33322f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.564566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bbf88fbe-f20d-4c20-b3ba-3aa27a33322f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:02.565895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bbf88fbe-f20d-4c20-b3ba-3aa27a33322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:02.589122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736cbb-0c1d-480b-b9f9-fc2574ed917b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.637824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.638688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.639120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.639549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.640628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:02.641111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.643351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.651550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 474/938] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:02.652957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.671267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 465/939] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:02.671948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf88fbe-f20d-4c20-b3ba-3aa27a33322f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.726539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:02.823997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf88fbe-f20d-4c20-b3ba-3aa27a33322f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.844611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910898db-700e-4a5b-819c-35f4dba6633c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:02.850953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:02.865637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.866265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.866533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.866815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.867457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:02.867758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.869220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.874681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 466/940] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:02.875545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:02.915493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:02.916405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:02.916907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:02.917387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:02.920942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:02.921468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:02.925170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:02.948714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 475/941] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:02.949952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.016008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434ce21-8f6c-4fee-b5f6-08ace75c0025 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.059497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:03.060808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.061435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.061705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.061988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.062732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] RBAC: Adding query filter params (interface=public, service_id=538d4c40516d48668e9410e328606b7c, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:03.063034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.064557 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.066449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943539c2-1260-4021-ad7b-de2b97deb65e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.070691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 467/942] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/endpoints?service_id=538d4c40516d48668e9410e328606b7c&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:03.071581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.107738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.108362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.108687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.108959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.109587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:03.109886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.111316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.116982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 476/943] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:49:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:03.117703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.143488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:03.242153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3af36c-650d-49fd-a317-2a5bfea66d8c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.252145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb484a0f-7b35-4dc0-9ff2-d0aca71cf5fb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.287998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.288682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.288950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.289225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.289896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.292133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.295297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING keystone.server.flask.common [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 17:49:03.313285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.314419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.314719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.315088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.317238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 17:49:03.317238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 17:49:03.317238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [parameters: {'id': '17bf8100aae64b46817f176c1bebe27b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 17:49:03.317238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17bf8100aae64b46817f176c1bebe27b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17bf8100aae64b46817f176c1bebe27b. Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 17:49:03.324514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 17:49:03.325415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Sep 13 17:49:03.326426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 17:49:03.327330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 17:49:03.328244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 17:49:03.332582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'id': '17bf8100aae64b46817f176c1bebe27b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:03.334728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:03.336685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17bf8100aae64b46817f176c1bebe27b. Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 468/944] 10.222.0.43 () {60 vars in 1140 bytes} [Tue Sep 13 17:49:03 2022] POST /identity/v3/projects => generated 262 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:03.339880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.341906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.341906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:49:03.355700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:03.360839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:03.366290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 477/945] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:49:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:03.369732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.403623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:03.469758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99821bd9-2026-46cb-a7f0-722fcd7b63c3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.484765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:03.511397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f371-5cb4-40e0-b27b-d986c3aaaba4 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.524184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.525197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.525629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.526030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.526953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.528958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.541189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 469/946] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:49:03 2022] GET /identity/v3/services => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:49:03.542150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.577881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.580572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.581455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.582584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.599889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.603475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:49:03.630965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:03.632760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:03.634465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:03.637321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 478/947] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:03.644165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.769951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a62f7-8259-4306-be5f-aa9519514f99 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.813299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:03.817026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.817613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.817890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.818198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.818971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:03.819270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.820819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.856273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddf030-5661-4db4-9fd5-8fe6dc64cd85 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:03.864468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 470/948] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 13 17:49:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:03.868090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:03.905479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:03.937793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:03.939257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:03.939739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:03.940209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:03.941442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:03.942193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:03.944909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:03.984532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 479/949] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:49:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:03.999997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b5c36-4ca5-4c4f-9bfb-53fd043d511d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:04.042493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:04.043163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:04.043455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:04.043707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:04.056178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:04.057723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:04.075316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:04.077221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:04.078953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:04.078953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:04.078953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:04.082251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 471/950] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:04.083360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:04.211663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e59835-6042-46a4-83ae-b53499214f7b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:04.250789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:04.251430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:04.251706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:04.251977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:04.252676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:04.252976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:04.254417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:04.264857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:04.268617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 480/951] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:04.275302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:04.376959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:04.398658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b71df-3fca-4954-8a0f-f5191a5b8694 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:04.435558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:04.436209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:04.436520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:04.436813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:04.444654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:04.446144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:49:04.459178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:04.461672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:04.463871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 472/952] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:04.476177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:04.600704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec0d78-4ef6-43fa-b1c5-a5ecb0fddc52 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:04.645740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30dde90-6ab5-4521-9a2e-e6ab8aa5af70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:04.646418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dde90-6ab5-4521-9a2e-e6ab8aa5af70 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:04.646699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dde90-6ab5-4521-9a2e-e6ab8aa5af70 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:04.646975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30dde90-6ab5-4521-9a2e-e6ab8aa5af70 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:04.647683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dde90-6ab5-4521-9a2e-e6ab8aa5af70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:04.647986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dde90-6ab5-4521-9a2e-e6ab8aa5af70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:04.649510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30dde90-6ab5-4521-9a2e-e6ab8aa5af70 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:04.665351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 481/953] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:49:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:04.675643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:04.855563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57a800-7c3a-466a-b8e8-1ef05384fe26 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:04.892249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea0351-31b2-4035-a0dc-fb32cbd20658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:04.892969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea0351-31b2-4035-a0dc-fb32cbd20658 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:04.893234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea0351-31b2-4035-a0dc-fb32cbd20658 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:04.893495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea0351-31b2-4035-a0dc-fb32cbd20658 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/1e6a922a2aea4d3ea1feadbd25cb32cf/roles/b4532c8703f94b348ec0e808e4543d64` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:04.895940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea0351-31b2-4035-a0dc-fb32cbd20658 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=1e6a922a2aea4d3ea1feadbd25cb32cf, role_id=b4532c8703f94b348ec0e808e4543d64)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:04.897400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea0351-31b2-4035-a0dc-fb32cbd20658 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:04.909094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:04.914507 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 473/954] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:49:04 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/1e6a922a2aea4d3ea1feadbd25cb32cf/roles/b4532c8703f94b348ec0e808e4543d64 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:49:06.011243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb98634-d348-40e1-af13-0b978208bf47 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.012146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb98634-d348-40e1-af13-0b978208bf47 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.012146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb98634-d348-40e1-af13-0b978208bf47 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.013837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 482/955] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:06.020118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030cdf-d11c-4d36-8c19-e94b872f3478 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.020649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030cdf-d11c-4d36-8c19-e94b872f3478 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.020756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030cdf-d11c-4d36-8c19-e94b872f3478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.040955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9b030cdf-d11c-4d36-8c19-e94b872f3478 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:06.042576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b030cdf-d11c-4d36-8c19-e94b872f3478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:06.124615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 474/956] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:06.391428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb98634-d348-40e1-af13-0b978208bf47 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:06.476217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:06.553659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4fd79-e184-4eaa-b923-f804c881fe36 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.554326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4fd79-e184-4eaa-b923-f804c881fe36 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.554779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4fd79-e184-4eaa-b923-f804c881fe36 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.556773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 475/957] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:06.561421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb98634-d348-40e1-af13-0b978208bf47 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:06.563332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feaa43-3b11-4981-af63-f01de4026dfd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.563947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feaa43-3b11-4981-af63-f01de4026dfd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.564471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feaa43-3b11-4981-af63-f01de4026dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.589316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-83feaa43-3b11-4981-af63-f01de4026dfd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:06.591079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-83feaa43-3b11-4981-af63-f01de4026dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:06.620111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42df240-0221-4fcd-8b36-93a15f33ebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:06.621214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df240-0221-4fcd-8b36-93a15f33ebc3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.621844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df240-0221-4fcd-8b36-93a15f33ebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.622341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42df240-0221-4fcd-8b36-93a15f33ebc3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.623295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42df240-0221-4fcd-8b36-93a15f33ebc3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:06.624757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42df240-0221-4fcd-8b36-93a15f33ebc3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b42df240-0221-4fcd-8b36-93a15f33ebc3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:06.635393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:06.642713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:06.648064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 483/958] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:49:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:06.650329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459c96-5c56-4c68-b802-2787e53b7c38 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.650329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459c96-5c56-4c68-b802-2787e53b7c38 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.650329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459c96-5c56-4c68-b802-2787e53b7c38 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.650329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 484/959] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:06.650329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f49f-876a-4d1c-80e7-2b45d000e0d5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.650329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f49f-876a-4d1c-80e7-2b45d000e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.651077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f49f-876a-4d1c-80e7-2b45d000e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.667771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9da6f49f-876a-4d1c-80e7-2b45d000e0d5 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:06.669103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9da6f49f-876a-4d1c-80e7-2b45d000e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:06.716695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 476/960] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:06.718242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feaa43-3b11-4981-af63-f01de4026dfd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:06.768678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 485/961] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:06.811030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:06.860683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feaa43-3b11-4981-af63-f01de4026dfd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:06.902767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:06.903295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:06.903568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:06.903832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:06.904505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:06.904875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:06.906297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:06.912354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 477/962] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:06.920934 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f49f-876a-4d1c-80e7-2b45d000e0d5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:07.015925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:07.084663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f49f-876a-4d1c-80e7-2b45d000e0d5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:07.148673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:07.149717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:07.150207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:07.150695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:07.151857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:07.152363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:07.153878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:07.159835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 486/963] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:07.169117 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:07.198266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:07.297456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec9660-e8e1-4bb6-9984-ea0c8f5c3b94 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:07.339071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:07.348181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140d165-9fb7-4e46-a200-01e11e0258ba None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:07.355096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:07.355875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:07.356291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:07.356729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:07.357861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:07.359405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:07.370714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:07.387601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 478/964] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:49:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:49:07.389161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:07.412050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:07.413656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:07.414423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:07.415179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:07.416660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:07.419643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:07.447206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 487/965] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:49:07 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:49:07.582298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f5be9-7638-4852-a17b-0321d3370c7d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:07.637118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:07.637858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:07.638132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:07.638397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:07.639297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:07.640855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:07.659687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 17:49:07.659687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 17:49:07.659687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [parameters: {'user_id': '1cd169852c5241cab76e5e54ec6ec740', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 17:49:07.659687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Sep 13 17:49:07.666910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 17:49:07.667908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 17:49:07.668936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 17:49:07.669839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 17:49:07.670749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 17:49:07.671642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'user_id': '1cd169852c5241cab76e5e54ec6ec740', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:07.673780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 17:49:07.676242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 17:49:07.679892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 17:49:07.679892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 17:49:07.679892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 17:49:07.679892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 17:49:07.679892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 17:49:07.679892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:07.679892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 479/966] 10.222.0.43 () {60 vars in 1131 bytes} [Tue Sep 13 17:49:07 2022] POST /identity/v3/users => generated 160 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:07.681988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:07.855207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd3b61-9121-4415-8c3d-67741a39ab16 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:07.924540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc618b-77cd-4396-ac5b-612ee43aa72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:07.925418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc618b-77cd-4396-ac5b-612ee43aa72a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:07.925880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc618b-77cd-4396-ac5b-612ee43aa72a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:07.926361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc618b-77cd-4396-ac5b-612ee43aa72a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:07.941295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc618b-77cd-4396-ac5b-612ee43aa72a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:07.943996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc618b-77cd-4396-ac5b-612ee43aa72a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-10dc618b-77cd-4396-ac5b-612ee43aa72a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:49:07.966124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:07.967157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 17:49:07.968109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:07.968700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 488/967] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:49:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:07.978251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:07.999879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c4e1b-71a5-4e5f-a88e-1a78f15988eb None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:08.000236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c4e1b-71a5-4e5f-a88e-1a78f15988eb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:08.000447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c4e1b-71a5-4e5f-a88e-1a78f15988eb None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:08.002135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 489/968] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:49:08.007839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a4cf1-6603-4c72-9804-f2eb7cada2d8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:08.008129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a4cf1-6603-4c72-9804-f2eb7cada2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:08.008380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a4cf1-6603-4c72-9804-f2eb7cada2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:08.040629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a71a4cf1-6603-4c72-9804-f2eb7cada2d8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:08.042830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a71a4cf1-6603-4c72-9804-f2eb7cada2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:08.053884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:08.093119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:08.158669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 490/969] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:08.181880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632c809-98cf-456c-be28-d77b4aca88cc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:08.248209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:08.249372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:08.249884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:08.250353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:08.251728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:08.252244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:08.255442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:08.328950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 480/970] 10.222.0.43 () {56 vars in 1101 bytes} [Tue Sep 13 17:49:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:08.425502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a4cf1-6603-4c72-9804-f2eb7cada2d8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:08.637666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a4cf1-6603-4c72-9804-f2eb7cada2d8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:08.709753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:08.710839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:08.711319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:08.711796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:08.733745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:08.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:08.745628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:08.747925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:08.750093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 491/971] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 13 17:49:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:08.758658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:08.895972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:08.908366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6f6e3-b970-47dd-954c-8e7709b9a915 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:08.979172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:08.980910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:08.981720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:08.982481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:08.984063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:08.984796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:08.987771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:09.005799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 481/972] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 13 17:49:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:09.016032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:09.065319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:09.202003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba265a6-a589-4c91-98b2-8b57cd7a3ca8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:09.263798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:09.264926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:09.265420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:09.265906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:09.284907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:09.287803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:09.316944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:09.317991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:09.318899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:09.318899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:09.318899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:09.319504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 492/973] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:09.328881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:09.522121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc272-2b78-470c-b270-6cb38743e044 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:09.562643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:09.563267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:09.563544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:09.563812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:09.564492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:09.564807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:09.566299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:09.583096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 482/974] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:09.589607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:09.739204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:09.783673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f34b3-f140-48c4-b7c8-d2e5068d3246 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:09.854369 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea31e0-8f29-4493-97f1-5a3bad804c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:09.855776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea31e0-8f29-4493-97f1-5a3bad804c14 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:09.856258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea31e0-8f29-4493-97f1-5a3bad804c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:09.856871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea31e0-8f29-4493-97f1-5a3bad804c14 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:09.872384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea31e0-8f29-4493-97f1-5a3bad804c14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:09.875187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea31e0-8f29-4493-97f1-5a3bad804c14 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4fea31e0-8f29-4493-97f1-5a3bad804c14 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:49:09.898419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:09.899469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:09.900941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 493/975] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:49:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:09.906921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:09.929204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:09.955753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a79cf-707e-45cb-9d5f-fff0289ff1c7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:09.956024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a79cf-707e-45cb-9d5f-fff0289ff1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:09.956289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a79cf-707e-45cb-9d5f-fff0289ff1c7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:09.958170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 494/976] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:49:09.964701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93095459-f2cf-4726-8a01-9a47030b2fab None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:09.965011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93095459-f2cf-4726-8a01-9a47030b2fab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:09.965347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93095459-f2cf-4726-8a01-9a47030b2fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:09.996171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-93095459-f2cf-4726-8a01-9a47030b2fab None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:10.000417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-93095459-f2cf-4726-8a01-9a47030b2fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:10.036612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dfa93-dab5-42ec-aa1a-77f2ad25af3e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:10.078763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:10.079968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:10.080497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:10.081358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:10.082247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:10.082795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:10.085510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:10.096791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 495/977] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:10.100711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422fe62-541b-41e3-b238-e703d3508a7e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:10.101005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422fe62-541b-41e3-b238-e703d3508a7e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:10.101279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422fe62-541b-41e3-b238-e703d3508a7e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:10.102627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 496/978] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:49:10.111625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab6a65-8165-40aa-a0c6-71c9cc5d0d74 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:10.112075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab6a65-8165-40aa-a0c6-71c9cc5d0d74 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:10.112623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab6a65-8165-40aa-a0c6-71c9cc5d0d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:10.129138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 483/979] 10.222.0.43 () {56 vars in 1149 bytes} [Tue Sep 13 17:49:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:10.134675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:10.143409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7fab6a65-8165-40aa-a0c6-71c9cc5d0d74 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:10.145458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7fab6a65-8165-40aa-a0c6-71c9cc5d0d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:10.239527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:10.243825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:10.255183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 497/980] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:10.280781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d405-78f5-4066-b27e-602937636ccc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:10.320560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:10.321075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:10.321353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:10.321604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:10.364624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab6a65-8165-40aa-a0c6-71c9cc5d0d74 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:10.412585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:10.414070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:10.491382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab6a65-8165-40aa-a0c6-71c9cc5d0d74 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:10.538144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:10.538568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a0581-5055-4386-9503-77c0780a902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:10.539585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a0581-5055-4386-9503-77c0780a902c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:10.540031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a0581-5055-4386-9503-77c0780a902c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:10.540905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a0581-5055-4386-9503-77c0780a902c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:10.542026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a0581-5055-4386-9503-77c0780a902c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:10.544838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a0581-5055-4386-9503-77c0780a902c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-217a0581-5055-4386-9503-77c0780a902c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:10.560814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:10.561893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:10.562812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:10.563567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 498/981] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:10.572273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a0581-5055-4386-9503-77c0780a902c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:10.581502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 484/982] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:10 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:49:10.590830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:10.690877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:10.760582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a0581-5055-4386-9503-77c0780a902c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:10.782633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080daaf-480d-4d4f-8d39-c5e440fa4f88 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:10.802013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:10.802698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:10.802974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:10.803248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:10.803926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:10.804257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:10.805684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:10.811464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 499/983] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:49:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:10.812061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:10.830844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:10.831418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:10.831710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:10.831978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:10.843277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:10.844816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:10.860706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:10.861955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:10.862880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:10.862880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:10.862880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:10.862880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 485/984] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:10.863113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:10.899854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:10.975531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4f02-d843-45f2-9bbc-fd565e6f3441 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:10.992930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500316-67e6-4a3f-a367-23ab23e158d9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.004462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:11.019385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.020303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.020734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.021253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.022062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.024269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.038530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.039164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:11.039331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:11.041307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:11.043058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:11.044643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.044643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.044643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:11.044643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.045136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 500/985] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:49:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:11.045136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.047101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.053625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 486/986] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:49:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:11.055557 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.181303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05e656-31ee-438b-b7e6-956ddcf2ff97 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.190646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:11.201643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74b1c-2b1e-41fb-9562-9aaca564d507 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.235152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.236214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.236745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.237216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.238407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:11.238951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.239531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.240301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.240671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.241039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.241574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.241849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:11.242212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.243731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.250614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 487/987] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:11.251260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.267031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 501/988] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:11.268487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.330725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:11.382271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0cb00-b739-4f6d-904c-b10a037f8fe5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.392028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa868ee-dbb6-4c4f-b0e1-4d9ceb27d202 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.424885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.425765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.425765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.426073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.426752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] RBAC: Adding query filter params (interface=public, service_id=a9a5be07c2104b77a1ff4c4edf577eac, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:11.427048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.428573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.433322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.433977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.434236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 488/989] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/endpoints?service_id=a9a5be07c2104b77a1ff4c4edf577eac&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:11.434415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.434787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.435078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.435786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:11.436205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.437846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.452506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 502/990] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:11.452506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.467960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:11.555789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382aee09-6eaa-40a7-9404-9af3fef7a298 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.565928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea7a16-ee2a-4b43-b12c-fedca23d36b5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.571479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:11.596402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.597053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.597330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.597598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.598208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.599592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.606090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 503/991] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:49:11.606878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.608583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.609600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.610034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.610456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.625501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.628040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:49:11.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:11.649891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:11.650816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 489/992] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:11.657451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.740317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcf33a-a4aa-4196-a1f4-decfd39f0fc5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.783752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.784830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.785348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.785662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:11.785835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.786409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] RBAC: Adding query filter params (interface=admin, service_id=538d4c40516d48668e9410e328606b7c, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:11.787298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.788386 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.803809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 504/993] 10.222.0.43 () {56 vars in 1249 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/endpoints?service_id=538d4c40516d48668e9410e328606b7c&interface=admin®ion_id=RegionOne => generated 193 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:11.806372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.818946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702da20-a17d-4abc-94e6-8d19e4b1590a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:11.873301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:11.874313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:11.877046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:11.877046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:11.877046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:11.877046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:11.879669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:11.884204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:11.904371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 490/994] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:11.914604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:11.980248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df76ed-986c-48f0-a7bd-aaf5ba3ba94b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:12.021977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22618b93-9e83-4c6c-ba20-f14647974eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:12.022974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22618b93-9e83-4c6c-ba20-f14647974eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:12.023267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22618b93-9e83-4c6c-ba20-f14647974eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:12.023541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22618b93-9e83-4c6c-ba20-f14647974eb7 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:12.024544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22618b93-9e83-4c6c-ba20-f14647974eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:12.026012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22618b93-9e83-4c6c-ba20-f14647974eb7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:12.040472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 505/995] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/services => generated 1727 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:49:12.095756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3df25-8b7b-4f80-88ee-cd90e4475abd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:12.157505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa925596-38e6-4644-8517-58599bf6305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:12.158612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa925596-38e6-4644-8517-58599bf6305f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:12.159180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa925596-38e6-4644-8517-58599bf6305f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:12.159712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa925596-38e6-4644-8517-58599bf6305f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:12.160899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa925596-38e6-4644-8517-58599bf6305f None admin] RBAC: Adding query filter params (role.id=b4532c8703f94b348ec0e808e4543d64, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=1e6a922a2aea4d3ea1feadbd25cb32cf) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:12.161543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa925596-38e6-4644-8517-58599bf6305f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:12.164288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa925596-38e6-4644-8517-58599bf6305f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:12.178317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 491/996] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:11 2022] GET /identity/v3/role_assignments?user.id=1e6a922a2aea4d3ea1feadbd25cb32cf&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=b4532c8703f94b348ec0e808e4543d64 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:14.244684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6973ce-ad58-437d-827b-dfa66a1623b3 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.245305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6973ce-ad58-437d-827b-dfa66a1623b3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.245554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6973ce-ad58-437d-827b-dfa66a1623b3 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.248070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 506/997] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:14.254186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6699de7-45a8-4857-a729-15fb60de91cd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.254917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6699de7-45a8-4857-a729-15fb60de91cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.254917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6699de7-45a8-4857-a729-15fb60de91cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.262951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:14.275857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e6699de7-45a8-4857-a729-15fb60de91cd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:14.277477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6699de7-45a8-4857-a729-15fb60de91cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:14.278383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092f319-abf4-4121-9b9d-8eac6419b41d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.278671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092f319-abf4-4121-9b9d-8eac6419b41d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.278936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092f319-abf4-4121-9b9d-8eac6419b41d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.280630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 507/998] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:14.286088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad879-185b-4d6a-94b3-4bbc43958b27 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.286372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad879-185b-4d6a-94b3-4bbc43958b27 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.286655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ad879-185b-4d6a-94b3-4bbc43958b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.306756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4d4ad879-185b-4d6a-94b3-4bbc43958b27 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:14.308001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4d4ad879-185b-4d6a-94b3-4bbc43958b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:14.373184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 492/999] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:14.398881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 508/1000] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:14.506901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9b7df-ff9f-4173-97ea-d88d8319c9a1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.507126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9b7df-ff9f-4173-97ea-d88d8319c9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.507374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9b7df-ff9f-4173-97ea-d88d8319c9a1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.508984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 493/1001] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:14.515019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44079cf-4935-43d2-8d7b-24b1ff28ef56 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.515310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44079cf-4935-43d2-8d7b-24b1ff28ef56 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.515584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44079cf-4935-43d2-8d7b-24b1ff28ef56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.535162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f44079cf-4935-43d2-8d7b-24b1ff28ef56 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:14.536379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f44079cf-4935-43d2-8d7b-24b1ff28ef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:14.567865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:14.613952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 509/1002] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:14.645432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9b7df-ff9f-4173-97ea-d88d8319c9a1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:14.787288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44079cf-4935-43d2-8d7b-24b1ff28ef56 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:14.794755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9b7df-ff9f-4173-97ea-d88d8319c9a1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:14.839179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:14.839846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.840118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.840388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.841032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:14.842498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:14.844972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:14.856903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:14.865063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:14.866780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 494/1003] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:14.869272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:14.952785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44079cf-4935-43d2-8d7b-24b1ff28ef56 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:14.990712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:14.991349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:14.991645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:14.991918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:14.992575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:14.993939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:15.003635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:15.005242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:15.006301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 510/1004] 10.222.0.43 () {56 vars in 1126 bytes} [Tue Sep 13 17:49:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:15.007245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.015069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af261d-d9c5-4144-92d0-2b5bffc8401e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.041175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:15.058030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.058929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.059366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.059796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.060866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:15.061286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.063587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.078381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 495/1005] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:49:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:15.078381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.134612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeaadc-662a-4351-ae1e-ba7be2a3ae83 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.157487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:15.175329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.176010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.176247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.176912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] PATH_INFO: `/v3/services/image` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.177145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.178574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:15.187771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:15.189728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:15.191690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 511/1006] 10.222.0.43 () {56 vars in 1105 bytes} [Tue Sep 13 17:49:15 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:15.193744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.209335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcca41-bcd2-4f67-904f-973ee9bb4b29 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.252645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.253268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.253545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.253815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.254470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:15.255028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.256222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.263934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 496/1007] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:15.265308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.337169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:15.355539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980eecc-4877-4aa1-9714-69d990365589 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.408011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedcf4e6-c1da-4261-befd-616aea218ef5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.411624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.412340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.412656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.412927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.413582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:15.413887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.415324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.426719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 512/1008] 10.222.0.43 () {56 vars in 1112 bytes} [Tue Sep 13 17:49:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:15.427549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.440354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:15.451105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.451993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.452416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.452924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.453757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] RBAC: Adding query filter params (name=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:15.454259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.455932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.462026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 497/1009] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:15.462981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.586849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace92c7-b811-4c24-aa74-a708bdad3a71 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.587475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b9135-63d7-405b-a104-f15984dc3fbb None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.632690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.633334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.633625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.633890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.634538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:15.634887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.637885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.643027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.643912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.644319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.644835 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.646097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.646284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 513/1010] 10.222.0.43 () {56 vars in 1122 bytes} [Tue Sep 13 17:49:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:15.647047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.648797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.683328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 498/1011] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:49:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:15.684025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:15.695574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:15.789112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:15.824536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e5c9b-18ea-4cbf-8a9d-5f5b3c8b20aa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.893701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d2bc4-4179-4e9c-98c3-951456d8b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.895024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d2bc4-4179-4e9c-98c3-951456d8b0ce None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.895024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d2bc4-4179-4e9c-98c3-951456d8b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.895024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d2bc4-4179-4e9c-98c3-951456d8b0ce None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.895542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d2bc4-4179-4e9c-98c3-951456d8b0ce None admin] RBAC: Adding query filter params (type=image) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:15.895673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d2bc4-4179-4e9c-98c3-951456d8b0ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.897173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d2bc4-4179-4e9c-98c3-951456d8b0ce None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:15.899029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cf3e7-45f4-4ad2-8e62-cb05889f210d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:15.903947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 514/1012] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:15.967602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214247a-4bfb-4730-8801-f49ccfbbc270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:15.968721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214247a-4bfb-4730-8801-f49ccfbbc270 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:15.969215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214247a-4bfb-4730-8801-f49ccfbbc270 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:15.969785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214247a-4bfb-4730-8801-f49ccfbbc270 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:15.971100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214247a-4bfb-4730-8801-f49ccfbbc270 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:15.973463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214247a-4bfb-4730-8801-f49ccfbbc270 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:16.003189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 499/1013] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:49:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:18.322686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6d23-24f1-49ff-9468-e562bbdba34d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:18.323509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6d23-24f1-49ff-9468-e562bbdba34d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:18.323509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad6d23-24f1-49ff-9468-e562bbdba34d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:18.326566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 515/1014] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:18.334511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaa05b-0aa3-42d2-a35a-c7048090a5c3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:18.335170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaa05b-0aa3-42d2-a35a-c7048090a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:18.335429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaa05b-0aa3-42d2-a35a-c7048090a5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:18.367053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7fcaa05b-0aa3-42d2-a35a-c7048090a5c3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:18.369228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7fcaa05b-0aa3-42d2-a35a-c7048090a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:18.510959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 500/1015] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:18 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:18.845704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc388-b16d-4c13-b848-5071fcdaa0fd None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:18.845957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc388-b16d-4c13-b848-5071fcdaa0fd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:18.846231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fc388-b16d-4c13-b848-5071fcdaa0fd None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:18.847943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 516/1016] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:18.854064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7f32-60df-4fcd-8e7d-bc0c10edbc95 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:18.854385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7f32-60df-4fcd-8e7d-bc0c10edbc95 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:18.854715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7f32-60df-4fcd-8e7d-bc0c10edbc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:18.875561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9d3f7f32-60df-4fcd-8e7d-bc0c10edbc95 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:18.876956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9d3f7f32-60df-4fcd-8e7d-bc0c10edbc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:18.906103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:18.950023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 501/1017] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:18 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:18.989994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fc388-b16d-4c13-b848-5071fcdaa0fd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:19.003910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fbd6-b5ad-4d1f-9529-fa39bab5d63e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.004242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fbd6-b5ad-4d1f-9529-fa39bab5d63e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.004500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fbd6-b5ad-4d1f-9529-fa39bab5d63e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.006187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 502/1018] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:19.011227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a2580-cf53-4a3d-bf20-adeb45d9fb12 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.012156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a2580-cf53-4a3d-bf20-adeb45d9fb12 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.012156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a2580-cf53-4a3d-bf20-adeb45d9fb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.037431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c76a2580-cf53-4a3d-bf20-adeb45d9fb12 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:19.039419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c76a2580-cf53-4a3d-bf20-adeb45d9fb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:19.092687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:19.136410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fc388-b16d-4c13-b848-5071fcdaa0fd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:19.140756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 503/1019] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:19 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:19.187969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:19.188651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.188954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.189204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.190406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:19.191857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:19.193916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 517/1020] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:19.199840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a2580-cf53-4a3d-bf20-adeb45d9fb12 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:19.213788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:19.324678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:19.336092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a2580-cf53-4a3d-bf20-adeb45d9fb12 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:19.352218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c55ef6-deca-4a2d-8927-424fc34c9504 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:19.375671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:19.376294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.376616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.376892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.377624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:19.379053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:19.420462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 504/1021] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:49:19 2022] POST /identity/v3/projects => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:19.420706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:19.420706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.420706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.420706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.433864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:19.438661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:19.440922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:19.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:19.445254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:19.448773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 518/1022] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 13 17:49:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:19.451038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:19.472677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:19.563440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45fc83-0ecc-48dc-8762-4f8bf84cc2d1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:19.604758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:19.605301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.605582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.605854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.606613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:19.608056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:19.627603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc584d2-07b3-4234-b58a-dc41aea7cdc5 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:19.630617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 505/1023] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:49:19 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:49:19.687006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:19.687999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.688868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.688868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.690003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:19.690492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:19.693067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:19.710317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 519/1024] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 13 17:49:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:19.718080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:19.724965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:19.904891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22ca7c-c09d-4806-80f4-a5655f638cbb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:19.963574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:19.964712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:19.965195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:19.965668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:19.984944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:19.987575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:20.014834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:20.015888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:20.017061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:20.017061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:20.017061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:20.017237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 506/1025] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:20.025110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:20.139806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:20.197967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949f2d5-0f44-428e-a074-c5b3750d13d7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:20.246464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:20.247095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:20.247373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:20.247639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:20.248523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:20.248844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:20.250304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:20.265857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 520/1026] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:20.272745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:20.376887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:20.465120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08b6f6-5af7-4b05-af96-7a877a89c765 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:20.526114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:20.527180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:20.527657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:20.528104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:20.542478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:20.545285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:49:20.567518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:20.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:20.575124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 507/1027] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:49:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:20.576670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:20.752100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea5d3-42ce-4a7f-9192-de3710616e7b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:20.790156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:20.801259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:20.801878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:20.802158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:20.802471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:20.803221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:20.803532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:20.805106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:20.829139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 521/1028] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:49:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:20.836477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:20.958985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071a3dd-8989-4f48-8ddc-6e445220b7e3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:20.977819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:21.000072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:21.000662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.000933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.001205 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.076529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:21.078196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:21.197538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 508/1029] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 17:49:21.203283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:21.344562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c193e4-cd36-47bb-9303-c5d93cbc9786 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:21.403236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:21.404118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.404446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.404723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.422749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:21.425544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:21.457719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:21.459597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:21.460805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:21.460805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:21.460805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:21.460805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 522/1030] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:21.470031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:21.495307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:21.628867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b80dd-740c-444e-ab84-0be3a6a56f10 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:21.677360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700fa69-2eaf-4899-9059-c8fa0bb3ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:21.683206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700fa69-2eaf-4899-9059-c8fa0bb3ef02 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.683206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700fa69-2eaf-4899-9059-c8fa0bb3ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.683206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700fa69-2eaf-4899-9059-c8fa0bb3ef02 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.683206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700fa69-2eaf-4899-9059-c8fa0bb3ef02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:21.683206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700fa69-2eaf-4899-9059-c8fa0bb3ef02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:21.683854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700fa69-2eaf-4899-9059-c8fa0bb3ef02 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:21.704374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 509/1031] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:21.713182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:21.744353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acdd34-054a-4e82-872b-bc056074face None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.744697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acdd34-054a-4e82-872b-bc056074face None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.744940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acdd34-054a-4e82-872b-bc056074face None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.748478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 510/1032] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:49:21.752323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b9258-3ba9-4a10-b43d-86874d927d3b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.752838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b9258-3ba9-4a10-b43d-86874d927d3b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.753270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b9258-3ba9-4a10-b43d-86874d927d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.773382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d77b9258-3ba9-4a10-b43d-86874d927d3b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:21.774854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d77b9258-3ba9-4a10-b43d-86874d927d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:21.784685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:21.882822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 511/1033] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:21.917224 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1040d73-b7df-4e48-80e1-60fb85959264 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.917634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1040d73-b7df-4e48-80e1-60fb85959264 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.917932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1040d73-b7df-4e48-80e1-60fb85959264 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.919325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 512/1034] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:21.928332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacadff-41c2-4321-9617-7dc59c7fd136 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.928672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacadff-41c2-4321-9617-7dc59c7fd136 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.929105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacadff-41c2-4321-9617-7dc59c7fd136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.929898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f33fb-8e2a-4a33-8dd0-c009286a51c3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:21.947238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:21.954906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cdacadff-41c2-4321-9617-7dc59c7fd136 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:21.956559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cdacadff-41c2-4321-9617-7dc59c7fd136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:21.970405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:21.971071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:21.971350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:21.971619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:21.981018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:21.984584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:49:21.999864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:22.006260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 523/1035] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:22.007252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:22.051704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 513/1036] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:22.118643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:22.125368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25040b38-551d-4e27-8431-a8628f38299d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:22.164892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:22.165419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:22.165698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:22.165961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:22.166682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:22.166969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:22.168409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:22.181771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 524/1037] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:49:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:22.188338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacadff-41c2-4321-9617-7dc59c7fd136 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:22.327515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:22.336124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacadff-41c2-4321-9617-7dc59c7fd136 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:22.376728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:22.377873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:22.378143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:22.378419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:22.379149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] RBAC: Adding query filter params (role.id=b4532c8703f94b348ec0e808e4543d64, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=90f40fe8f20f4a5bbd201debda6a97a5) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:22.379454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:22.381051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:22.384221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:22.387955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 514/1038] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:22 2022] GET /identity/v3/role_assignments?user.id=90f40fe8f20f4a5bbd201debda6a97a5&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=b4532c8703f94b348ec0e808e4543d64 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:22.389062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:22.470100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f5125-b611-4fe8-bd88-1996eb38acd1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:22.514364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:22.515024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:22.515305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:22.515562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:22.516155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:22.517617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:22.528043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:22.529581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:22.530665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:22.534618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 525/1039] 10.222.0.43 () {56 vars in 1105 bytes} [Tue Sep 13 17:49:22 2022] GET /identity/v3/services/image => generated 86 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:22.536086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:22.561319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc853182-a21e-45a2-b4f3-1eb4fe16495d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:22.578475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:22.605196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:22.605843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:22.606118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:22.606377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:22.607573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:22.609644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:22.611470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 515/1040] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:22.617470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:22.697415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d7ced-8777-47f9-b287-0070a7dc50d9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:22.704956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:22.742784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:22.743562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:22.743698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:22.744122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:22.745182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:22.745651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:22.747069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:22.753054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 526/1041] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:22.755765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454fa0-777a-42a7-bfbf-981713cedb37 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:22.759179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:22.809777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:22.810531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:22.810897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:22.811174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:22.812008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:22.814046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:22.843977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 516/1042] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:49:22 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:49:22.879434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:22.905111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537d9f2-7e10-4395-90d1-91fa681a96f8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:22.968854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:22.969923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:22.970366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:22.970918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:22.972073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] RBAC: Adding query filter params (type=image) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:22.972651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:22.975354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:22.990207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 527/1043] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:22.993854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:23.146947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7729ca-b03a-48b6-9fc5-314150f62a1f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:23.175428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:23.191049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ee848-5550-473d-a01a-d7fb186f0de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:23.191687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ee848-5550-473d-a01a-d7fb186f0de8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:23.191966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ee848-5550-473d-a01a-d7fb186f0de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:23.192240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ee848-5550-473d-a01a-d7fb186f0de8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:23.193099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ee848-5550-473d-a01a-d7fb186f0de8 None admin] RBAC: Adding query filter params (interface=public, service_id=e3b44d83187a44abb42e6741fc7234d0, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:23.193404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ee848-5550-473d-a01a-d7fb186f0de8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:23.194849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ee848-5550-473d-a01a-d7fb186f0de8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:23.204461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 517/1044] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:49:22 2022] GET /identity/v3/endpoints?service_id=e3b44d83187a44abb42e6741fc7234d0&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:23.210530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:23.383598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4e5c3-aa45-4150-b7ac-e18037d4e770 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:23.443965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7460-ba5a-479c-beaf-f8573e9dc13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:23.444699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7460-ba5a-479c-beaf-f8573e9dc13e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:23.444977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7460-ba5a-479c-beaf-f8573e9dc13e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:23.445243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7460-ba5a-479c-beaf-f8573e9dc13e None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:23.445868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac7460-ba5a-479c-beaf-f8573e9dc13e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:23.447331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac7460-ba5a-479c-beaf-f8573e9dc13e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:23.455304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 528/1045] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:49:23 2022] GET /identity/v3/services => generated 1957 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 17:49:24.680784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a191-89a9-49fa-9d2f-b95781808c5e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:24.680784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a191-89a9-49fa-9d2f-b95781808c5e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:24.681426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a191-89a9-49fa-9d2f-b95781808c5e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:24.683968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 518/1046] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:24.692096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6b23c-1be8-4daf-b0ad-77f5b4e7ce12 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:24.692614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6b23c-1be8-4daf-b0ad-77f5b4e7ce12 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:24.692854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6b23c-1be8-4daf-b0ad-77f5b4e7ce12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:24.713078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c5a6b23c-1be8-4daf-b0ad-77f5b4e7ce12 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:24.714293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c5a6b23c-1be8-4daf-b0ad-77f5b4e7ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:24.724491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:24.804405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 529/1047] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:25.245763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6a191-89a9-49fa-9d2f-b95781808c5e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:25.377570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6a191-89a9-49fa-9d2f-b95781808c5e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:25.416284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7d5b-a168-4e32-b2f4-e4b15546641e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:25.416627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7d5b-a168-4e32-b2f4-e4b15546641e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:25.416851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7d5b-a168-4e32-b2f4-e4b15546641e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:25.417066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:25.417776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:25.418184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:25.418559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 530/1048] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:49:25.418681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:25.425137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587eb0-95a6-422e-a0ce-d317cb74bdce None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:25.425728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587eb0-95a6-422e-a0ce-d317cb74bdce None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:25.426223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587eb0-95a6-422e-a0ce-d317cb74bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:25.435509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:25.440058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:25.441615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:25.442666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 519/1049] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 13 17:49:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:25.447989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:25.453275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-63587eb0-95a6-422e-a0ce-d317cb74bdce None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:25.455166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-63587eb0-95a6-422e-a0ce-d317cb74bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:25.525388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:25.563468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 531/1050] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:25.597363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f29407-31e7-4eaa-975b-c613cf6e82bd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:25.646039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:25.646728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:25.647101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:25.647367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:25.648342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:25.648816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:25.650331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:25.668458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 520/1051] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 13 17:49:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:25.670640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:25.670908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d864be-7a78-4364-9a9e-3dc310993db1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:25.671126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d864be-7a78-4364-9a9e-3dc310993db1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:25.671405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d864be-7a78-4364-9a9e-3dc310993db1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:25.673605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 532/1052] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:25.678767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da341-36a8-4168-bd6c-5db700e9cd25 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:25.679148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da341-36a8-4168-bd6c-5db700e9cd25 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:25.679316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da341-36a8-4168-bd6c-5db700e9cd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:25.698329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-704da341-36a8-4168-bd6c-5db700e9cd25 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:25.699532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-704da341-36a8-4168-bd6c-5db700e9cd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:25.779063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 533/1053] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:25.823940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa18408-6c82-4d17-b06c-a55c010426b9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:25.836734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:25.877050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:25.877847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:25.878286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:25.878754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:25.883928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da341-36a8-4168-bd6c-5db700e9cd25 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:25.887071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:25.897178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:25.898794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:25.914562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:25.915803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:25.916992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:25.916992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:25.916992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:25.916992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 521/1054] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:25.924814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.040748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da341-36a8-4168-bd6c-5db700e9cd25 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.073924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38c87e-552d-456e-89aa-9d4275d9f439 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.093053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.093694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.093969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.094247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.106263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:26.111156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:26.112532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 534/1055] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:49:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:26.113610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.121086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.121829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.122201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.122572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.123336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:26.123727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.125315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.143904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 522/1056] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:26.145322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.173171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:26.190653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:26.232845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534c7fd-a795-4ef5-a07e-2a5d0cc8e59b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.257056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95db4aa-1887-42e9-b23d-239c48d68461 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.269495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.270163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.270424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.270691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] PATH_INFO: `/v3/services/image` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.271327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.272832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:26.282293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:26.284590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:26.286276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 535/1057] 10.222.0.43 () {56 vars in 1105 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:26.287922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.316452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.317043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.317321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.317597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.318350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:26.318629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.320041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.329422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 523/1058] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:26.330323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.431631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873be9ae-3351-4dd0-8571-16f5dec4ccea None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.459230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:26.461916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:26.472707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.473354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.473639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.473911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.481013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e8775-75f1-4bb0-96da-6fc782786525 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.482957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.484474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:49:26.496709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:26.498193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 536/1059] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:26.499186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.526686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.527311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.527586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.527851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.528535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:26.528838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.530244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.535609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 524/1060] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:26.536477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.613796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1585-22b1-40e2-934d-a74d66ca85f2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.667948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.668856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.670095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.670095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] PATH_INFO: `/v3/users/83f2aceed46c4902966267fe049b8bb2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.677717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea8402-1dd9-4e70-bcdf-5861d9fbb7c0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.691186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] RBAC: Authorizing `identity:get_user(user_id=83f2aceed46c4902966267fe049b8bb2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.692863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.694836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 537/1061] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/users/83f2aceed46c4902966267fe049b8bb2 => generated 285 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:26.695724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.736317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.737163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.737549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.737910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.738830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:26.739241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.741570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.754791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:26.764141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 525/1062] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:26.765244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.779042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:26.835569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5d591-80bb-4fba-a03a-2f8d19e71618 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.885066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.885646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.885980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.886417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:26.887603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:26.887947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:26.889683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:26.896206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 538/1063] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:26.897319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:26.907246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579a10d-8ff1-4795-b486-c503bc5efd87 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:26.948859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:26.949582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:26.949864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:26.950223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.002480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efe0df-b1b3-46bf-a7f0-fc099fbc51e3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:27.012690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:27.048124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:27.049031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:27.049469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:27.049900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.054078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.056563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.066583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.069177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.070601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:27.089378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:27.090633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:27.091720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:27.091720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:27.091720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:27.091720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 539/1064] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:27.091720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:27.181197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 526/1065] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:49:27.181936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:27.201975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4779d20-48e9-4598-8645-3a5c0c3e9125 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:27.240390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:27.241036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:27.241318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:27.241587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.242275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.243668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.266368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 540/1066] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:49:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:27.267219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:27.289372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:27.325372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e880-0119-4178-a688-f60806ba3e7a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:27.335778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:27.397094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:27.398287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:27.398740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:27.399752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.400915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:27.401231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.403460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.424481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 527/1067] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:27.433586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39740a8c-6614-41a6-959f-583e69b64eb2 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:27.433795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:27.485838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:27.486558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:27.486841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:27.487091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.488383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] RBAC: Authorizing `identity:get_project(project_id=2fe586923be749ad968d000e5b733e36)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.489960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.491859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 541/1068] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:49:27 2022] GET /identity/v3/projects/2fe586923be749ad968d000e5b733e36 => generated 311 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:27.498216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:27.568733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b98d69-10e9-4b94-9b02-4896a6f809fe None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:27.603579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:27.604220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:27.604501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:27.604765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.612180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.613778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:49:27.626037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:27.627847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:27.629693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 528/1069] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:27.633662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:27.643822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:27.671817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:27.672094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f63cc-45aa-4c03-8527-fec0560d6388 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:27.741421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:27.742214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:27.742562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:27.743020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.743953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2fe586923be749ad968d000e5b733e36, user.id=83f2aceed46c4902966267fe049b8bb2) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:27.744284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.745774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.752373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 542/1070] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:27 2022] GET /identity/v3/role_assignments?user.id=83f2aceed46c4902966267fe049b8bb2&scope.project.id=2fe586923be749ad968d000e5b733e36&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:27.794093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd52c4-8af7-4909-823d-2a73ec67ff2c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:27.836281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d1afd-f6c7-40cc-ad5b-84f72a35b1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:27.837040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d1afd-f6c7-40cc-ad5b-84f72a35b1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:27.837322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d1afd-f6c7-40cc-ad5b-84f72a35b1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:27.837593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d1afd-f6c7-40cc-ad5b-84f72a35b1f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:27.838329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d1afd-f6c7-40cc-ad5b-84f72a35b1f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:27.838628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d1afd-f6c7-40cc-ad5b-84f72a35b1f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:27.840147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d1afd-f6c7-40cc-ad5b-84f72a35b1f1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:27.855247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 529/1071] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:49:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:27.861677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:28.019213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f25cbf-6ee5-44c7-8469-051aa92e24ca None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:28.067581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c52258-ec99-4388-a7c6-fa82a4f9b4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:28.068254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52258-ec99-4388-a7c6-fa82a4f9b4c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:28.068552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52258-ec99-4388-a7c6-fa82a4f9b4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:28.068822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c52258-ec99-4388-a7c6-fa82a4f9b4c6 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/90f40fe8f20f4a5bbd201debda6a97a5/roles/b4532c8703f94b348ec0e808e4543d64` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:28.071092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c52258-ec99-4388-a7c6-fa82a4f9b4c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=90f40fe8f20f4a5bbd201debda6a97a5, role_id=b4532c8703f94b348ec0e808e4543d64)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:28.072589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c52258-ec99-4388-a7c6-fa82a4f9b4c6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:28.095792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 543/1072] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:49:27 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/90f40fe8f20f4a5bbd201debda6a97a5/roles/b4532c8703f94b348ec0e808e4543d64 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:49:30.179881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734d5f-bdfe-4c2b-9af4-f19da9d437fe None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:30.180567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734d5f-bdfe-4c2b-9af4-f19da9d437fe None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:30.180567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa734d5f-bdfe-4c2b-9af4-f19da9d437fe None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:30.182116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 530/1073] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:30.188707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041f409-a8a0-4682-b372-c2ef2b967069 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:30.189441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041f409-a8a0-4682-b372-c2ef2b967069 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:30.189441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041f409-a8a0-4682-b372-c2ef2b967069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:30.209231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b041f409-a8a0-4682-b372-c2ef2b967069 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:30.210533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b041f409-a8a0-4682-b372-c2ef2b967069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:30.213212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:30.282559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f896e-4e71-413f-b24d-347f11270754 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:30.282959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f896e-4e71-413f-b24d-347f11270754 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:30.283135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f896e-4e71-413f-b24d-347f11270754 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:30.284752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 531/1074] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:49:30.290359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9b353-c26a-40b5-8f07-0b3ca979f499 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:30.290637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9b353-c26a-40b5-8f07-0b3ca979f499 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:30.290906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9b353-c26a-40b5-8f07-0b3ca979f499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:30.314695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1ea9b353-c26a-40b5-8f07-0b3ca979f499 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:30.316619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1ea9b353-c26a-40b5-8f07-0b3ca979f499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:30.327221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 544/1075] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:30.408488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 532/1076] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:30.754242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041f409-a8a0-4682-b372-c2ef2b967069 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:30.784727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e3dd-c02a-4a28-a9cf-183e9dc8b69c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:30.785041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e3dd-c02a-4a28-a9cf-183e9dc8b69c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:30.785266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e3dd-c02a-4a28-a9cf-183e9dc8b69c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:30.786889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 533/1077] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:30.792783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c5e3-5514-472c-b1d2-778e9f1960c0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:30.793050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c5e3-5514-472c-b1d2-778e9f1960c0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:30.793318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c5e3-5514-472c-b1d2-778e9f1960c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:30.814338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b624c5e3-5514-472c-b1d2-778e9f1960c0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:30.815561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b624c5e3-5514-472c-b1d2-778e9f1960c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:30.831294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:30.893329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041f409-a8a0-4682-b372-c2ef2b967069 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:30.924410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 534/1078] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:30.925349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624c5e3-5514-472c-b1d2-778e9f1960c0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:30.944650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:30.945584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:30.946077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:30.946553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:30.961228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:31.045830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624c5e3-5514-472c-b1d2-778e9f1960c0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.060158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:31.062084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:31.102729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.103682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.104091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.104540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.122355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:31.130851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:31.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:31.133108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 535/1079] 10.222.0.43 () {56 vars in 1120 bytes} [Tue Sep 13 17:49:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:31.139691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.144701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:31.208256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 545/1080] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:49:31.209523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.259416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:31.279868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f1450-83be-412f-96f8-bd8683225e78 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.325196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9539f1-611d-4122-baab-3a47f229c014 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.336720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.337731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.338211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.338666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.339829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:31.340355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:31.343911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:31.358385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 536/1081] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:31.359842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.367626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.368811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.369299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.369806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.387497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:31.392523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:31.393513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:31.394397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 546/1082] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:31.395395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.444548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:31.488191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb0574-7bda-4038-871e-ef32fae6f113 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.508908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:31.514624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fb963-3f4c-4ef2-bf96-04b7eb53e1cd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.535159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.535955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.536324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.536726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.549395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:31.551776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:31.559534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.560570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.561036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.561485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:31.573146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:31.574378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:31.575422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:31.575422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:31.575422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:31.575611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:31.575703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 537/1083] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:31.575775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:31.576638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:31.593113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:31.594241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:31.595621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:31.595621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:31.595621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:31.595621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 547/1084] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:31.596069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.704413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c26b-388a-4ade-b4a3-d07f909285b7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.722319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:31.740553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1fc35-f2a9-4764-bdda-081369cd6a1f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.745067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.745687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.745967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.746240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.746916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:31.747215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:31.748648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:31.758856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 538/1085] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:31.759716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.776033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.776679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.776944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.777214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.777869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:31.778160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:31.779561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:31.792392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 548/1086] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:31.793269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.830519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:31.890729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9e07-77a8-4aea-b2a5-19fb19698b9d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:31.930929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:31.931685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:31.931968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:31.932238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:31.932981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:31.933395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:31.934863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:31.948534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 539/1087] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:31.954389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:31.988090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a7c6a-878f-4a38-9726-d8ee9d3fbc40 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:32.010370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:32.037927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:32.038856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:32.039920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:32.039920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] PATH_INFO: `/v3/users/83f2aceed46c4902966267fe049b8bb2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:32.041952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] RBAC: Authorizing `identity:get_user(user_id=83f2aceed46c4902966267fe049b8bb2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:32.045421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:32.047801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 549/1088] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/users/83f2aceed46c4902966267fe049b8bb2 => generated 285 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:32.056128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:32.090801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89116eb-3c2a-4416-860d-b32c20235eea None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:32.134859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:32.135536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:32.135822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:32.136086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:32.148025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:32.150004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:32.158913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:49:32.162485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:32.164165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:32.165331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 540/1089] 10.222.0.43 () {56 vars in 1093 bytes} [Tue Sep 13 17:49:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:32.169658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:32.186206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591bc74-d56c-464d-ab9f-e5ec2568447d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:32.222326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:32.223029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:32.223243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:32.223243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:32.224924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] RBAC: Authorizing `identity:get_project(project_id=2fe586923be749ad968d000e5b733e36)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:32.226397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:32.228080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 550/1090] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:49:32 2022] GET /identity/v3/projects/2fe586923be749ad968d000e5b733e36 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:32.234368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:32.298341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:32.302963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fee98-f8ef-4e7a-84a0-b0fe114f8875 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:32.343742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:32.344265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:32.344710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:32.345024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:32.345812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:32.346236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:32.348120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:32.354864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d617f04-6820-4171-a6da-9534fb253d1f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:32.368778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 541/1091] 10.222.0.43 () {56 vars in 1133 bytes} [Tue Sep 13 17:49:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:32.374810 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:32.401185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54817a06-5287-4250-880b-791511725f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:32.401871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54817a06-5287-4250-880b-791511725f13 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:32.402152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54817a06-5287-4250-880b-791511725f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:32.402428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54817a06-5287-4250-880b-791511725f13 None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36/users/83f2aceed46c4902966267fe049b8bb2/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:32.404796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54817a06-5287-4250-880b-791511725f13 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fe586923be749ad968d000e5b733e36, user_id=83f2aceed46c4902966267fe049b8bb2, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:32.406261 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54817a06-5287-4250-880b-791511725f13 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:32.421531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 551/1092] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:49:32 2022] PUT /identity/v3/projects/2fe586923be749ad968d000e5b733e36/users/83f2aceed46c4902966267fe049b8bb2/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 17:49:32.515060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24065edd-4a66-41c6-ae8d-5b555f35bc61 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:32.556926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:32.557572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:32.557917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:32.558196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:32.588217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:32.640299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:32.641805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:32.754896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 542/1093] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 17:49:32.761043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54817a06-5287-4250-880b-791511725f13 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:32.771050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:32.870268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54817a06-5287-4250-880b-791511725f13 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:32.907204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:32.907773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:32.908046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:32.908317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:32.920826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:32.922450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:49:32.938282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:32.939552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:49:32.940635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:49:32.940635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:49:32.940635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:32.944465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 552/1094] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:49:32.945992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:33.033141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:33.106377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298a591-56f7-450c-989f-8ee356fdd0e3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:33.145493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:33.146248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:33.146597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:33.146936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:33.147684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:33.148064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:33.149705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:33.163790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 543/1095] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:49:33.170876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:33.249979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:33.296119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bc9fa-803e-4550-9f7c-c1697e77ad0b None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:33.332905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:33.333586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:33.334005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:33.334288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:33.342474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:33.344000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:49:33.357508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:33.358733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:49:33.359699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 553/1096] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:33.365350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:33.475172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765eea9a-d2e6-45c5-9600-39736e163690 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:33.492107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:33.513582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bbfbf-6091-419e-bd8d-ff3e35c49bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:33.514183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bbfbf-6091-419e-bd8d-ff3e35c49bde None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:33.514459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bbfbf-6091-419e-bd8d-ff3e35c49bde None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:33.514752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bbfbf-6091-419e-bd8d-ff3e35c49bde None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:33.515435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bbfbf-6091-419e-bd8d-ff3e35c49bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:33.515741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bbfbf-6091-419e-bd8d-ff3e35c49bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:33.517253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bbfbf-6091-419e-bd8d-ff3e35c49bde None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:33.531322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 544/1097] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:49:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:33.538043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:33.666187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a54bd4-db40-47ed-9e06-ff6eb633cb2e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:33.687581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:33.706519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886b5a2-ea63-41de-81d2-6f1a037b7a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:33.707150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886b5a2-ea63-41de-81d2-6f1a037b7a3b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:33.707431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886b5a2-ea63-41de-81d2-6f1a037b7a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:33.707700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886b5a2-ea63-41de-81d2-6f1a037b7a3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:33.708475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886b5a2-ea63-41de-81d2-6f1a037b7a3b None admin] RBAC: Adding query filter params (role.id=b4532c8703f94b348ec0e808e4543d64, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=90f40fe8f20f4a5bbd201debda6a97a5) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:33.708834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886b5a2-ea63-41de-81d2-6f1a037b7a3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:33.710332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886b5a2-ea63-41de-81d2-6f1a037b7a3b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:33.717518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 554/1098] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:33 2022] GET /identity/v3/role_assignments?user.id=90f40fe8f20f4a5bbd201debda6a97a5&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=b4532c8703f94b348ec0e808e4543d64 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:34.623242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16956096-5486-4f6f-90f4-0a8f98a869cd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:34.623841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16956096-5486-4f6f-90f4-0a8f98a869cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:34.623841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16956096-5486-4f6f-90f4-0a8f98a869cd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:34.625158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 545/1099] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:34.631140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336a52-03ad-4565-9736-2f85ea5febb0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:34.631922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336a52-03ad-4565-9736-2f85ea5febb0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:34.631922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336a52-03ad-4565-9736-2f85ea5febb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:34.659749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c9336a52-03ad-4565-9736-2f85ea5febb0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:34.661059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c9336a52-03ad-4565-9736-2f85ea5febb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:34.745005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 555/1100] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:49:35.008353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16956096-5486-4f6f-90f4-0a8f98a869cd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:35.125363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16956096-5486-4f6f-90f4-0a8f98a869cd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:35.163694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:35.164340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:35.164656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:35.164928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:35.176489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:35.181381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:35.182556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:35.183422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 546/1101] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:35.188939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9336a52-03ad-4565-9736-2f85ea5febb0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:35.306186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9336a52-03ad-4565-9736-2f85ea5febb0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:35.322286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:35.346027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:35.346643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:35.346919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:35.347165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:35.347861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:35.348163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:35.349641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:35.358645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 556/1102] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:35.365131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:35.404313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:35.481420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216eedb-dd87-4f9a-a99c-b4e524b365a1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:35.522241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:35.522880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:35.523155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:35.523428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] PATH_INFO: `/v3/users/83f2aceed46c4902966267fe049b8bb2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:35.524670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] RBAC: Authorizing `identity:get_user(user_id=83f2aceed46c4902966267fe049b8bb2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:35.526134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:35.527777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 547/1103] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:35 2022] GET /identity/v3/users/83f2aceed46c4902966267fe049b8bb2 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:35.534088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:35.645913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ab36e-3621-48a8-8806-788d58c8481f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:35.683304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:35.683933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:35.684201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:35.684493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:35.732171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:35.759778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:35.761316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:35.859901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 557/1104] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:49:35.866151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:35.971670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a967d-b70d-477a-a45e-cee775c4d3f3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:35.974042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:36.008915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1a51d-d48c-4102-97ed-efe7baf2ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:36.009531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a51d-d48c-4102-97ed-efe7baf2ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:36.009804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a51d-d48c-4102-97ed-efe7baf2ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:36.010066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1a51d-d48c-4102-97ed-efe7baf2ecc3 None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:36.011235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1a51d-d48c-4102-97ed-efe7baf2ecc3 None admin] RBAC: Authorizing `identity:get_project(project_id=2fe586923be749ad968d000e5b733e36)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:36.013050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1a51d-d48c-4102-97ed-efe7baf2ecc3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:36.014682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 548/1105] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:49:35 2022] GET /identity/v3/projects/2fe586923be749ad968d000e5b733e36 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:36.021339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:36.099073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:36.124580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba27c0-8b1d-4360-9980-75bb700280cd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:36.159703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e4e73-2111-4fb7-addd-b56f08bf582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:36.160308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4e73-2111-4fb7-addd-b56f08bf582e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:36.160611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4e73-2111-4fb7-addd-b56f08bf582e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:36.160879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4e73-2111-4fb7-addd-b56f08bf582e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:36.161609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e4e73-2111-4fb7-addd-b56f08bf582e None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2fe586923be749ad968d000e5b733e36, user.id=83f2aceed46c4902966267fe049b8bb2) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:36.161914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e4e73-2111-4fb7-addd-b56f08bf582e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:36.163343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e4e73-2111-4fb7-addd-b56f08bf582e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:36.169245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 558/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:36 2022] GET /identity/v3/role_assignments?user.id=83f2aceed46c4902966267fe049b8bb2&scope.project.id=2fe586923be749ad968d000e5b733e36&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:38.342796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2989b2-72ba-4144-80a1-6954e38638de None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:38.343388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2989b2-72ba-4144-80a1-6954e38638de None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:38.343388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2989b2-72ba-4144-80a1-6954e38638de None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:38.344814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 549/1107] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:38.351377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de65c7-48d6-4e4e-bcb4-c7b79fc8f92b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:38.351909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de65c7-48d6-4e4e-bcb4-c7b79fc8f92b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:38.351909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de65c7-48d6-4e4e-bcb4-c7b79fc8f92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:38.372045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f2de65c7-48d6-4e4e-bcb4-c7b79fc8f92b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:38.373326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f2de65c7-48d6-4e4e-bcb4-c7b79fc8f92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:38.449355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 559/1108] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:38.713525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2989b2-72ba-4144-80a1-6954e38638de None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:38.823194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2989b2-72ba-4144-80a1-6954e38638de None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:38.858675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1785a-c3b3-4030-895f-9089eb57a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:38.859301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1785a-c3b3-4030-895f-9089eb57a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:38.859566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1785a-c3b3-4030-895f-9089eb57a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:38.859829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1785a-c3b3-4030-895f-9089eb57a2b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:38.861003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1785a-c3b3-4030-895f-9089eb57a2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:38.862434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1785a-c3b3-4030-895f-9089eb57a2b1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:38.864017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 550/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:38.870337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de65c7-48d6-4e4e-bcb4-c7b79fc8f92b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:38.935964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:38.971470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de65c7-48d6-4e4e-bcb4-c7b79fc8f92b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:39.006917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26139ddc-1f46-4f8e-98ce-cf33dc4eef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:39.007525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26139ddc-1f46-4f8e-98ce-cf33dc4eef44 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:39.007832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26139ddc-1f46-4f8e-98ce-cf33dc4eef44 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:39.008098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26139ddc-1f46-4f8e-98ce-cf33dc4eef44 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:39.008780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26139ddc-1f46-4f8e-98ce-cf33dc4eef44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:39.010228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26139ddc-1f46-4f8e-98ce-cf33dc4eef44 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:39.032602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 560/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:49:38 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:41.204234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5038-ffb6-41a0-b997-6421657a9aea None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:41.204906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5038-ffb6-41a0-b997-6421657a9aea None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:41.204906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5038-ffb6-41a0-b997-6421657a9aea None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:41.206175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 551/1111] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:41.212585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc060d-c6da-42de-bae6-e367add5d7e3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:41.213130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc060d-c6da-42de-bae6-e367add5d7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:41.213130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc060d-c6da-42de-bae6-e367add5d7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:41.233347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fbfc060d-c6da-42de-bae6-e367add5d7e3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:41.234599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fbfc060d-c6da-42de-bae6-e367add5d7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:41.311009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 561/1112] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:41.578550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d5038-ffb6-41a0-b997-6421657a9aea None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:41.626168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:41.692514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d5038-ffb6-41a0-b997-6421657a9aea None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:41.727839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:41.731129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:41.731129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:41.731129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] PATH_INFO: `/v3/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:41.731129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] RBAC: Authorizing `identity:get_role(role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:41.731771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:41.733170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 552/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:41 2022] GET /identity/v3/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:41.739231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc060d-c6da-42de-bae6-e367add5d7e3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:41.794369 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:41.840962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc060d-c6da-42de-bae6-e367add5d7e3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:41.875986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:41.876655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:41.876935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:41.877206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] PATH_INFO: `/v3/users/99fbd98cf5b14462a8108692ba32eb19` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:41.891495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] RBAC: Authorizing `identity:get_user(user_id=99fbd98cf5b14462a8108692ba32eb19)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:41.893004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:41.894664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 562/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:41 2022] GET /identity/v3/users/99fbd98cf5b14462a8108692ba32eb19 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:41.900947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:42.006806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8add-93bd-42cb-9462-a47f5bbb370c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:42.017414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:42.045008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:42.045623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:42.045898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:42.046171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:42.118981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:42.120352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:42.220695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 553/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:49:42.227070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:42.341987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907197eb-deec-42f2-91f0-c21f927e521d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:42.358075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:42.385533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc7e20-fc48-40f4-a346-16400d0803c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:42.387480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7e20-fc48-40f4-a346-16400d0803c6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:42.387757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7e20-fc48-40f4-a346-16400d0803c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:42.388023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7e20-fc48-40f4-a346-16400d0803c6 None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:42.389251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc7e20-fc48-40f4-a346-16400d0803c6 None admin] RBAC: Authorizing `identity:get_project(project_id=2fe586923be749ad968d000e5b733e36)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:42.390701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc7e20-fc48-40f4-a346-16400d0803c6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:42.392334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 563/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:49:42 2022] GET /identity/v3/projects/2fe586923be749ad968d000e5b733e36 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:42.398904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:42.409695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:42.503522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cea660-b9c9-4290-bc14-e0274c924179 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:42.539671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac26350-a58e-4d39-b839-88a4d25dd4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:42.540281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac26350-a58e-4d39-b839-88a4d25dd4fe None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:42.540577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac26350-a58e-4d39-b839-88a4d25dd4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:42.540853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac26350-a58e-4d39-b839-88a4d25dd4fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:42.541585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac26350-a58e-4d39-b839-88a4d25dd4fe None admin] RBAC: Adding query filter params (role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=2fe586923be749ad968d000e5b733e36, user.id=99fbd98cf5b14462a8108692ba32eb19) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:42.541879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac26350-a58e-4d39-b839-88a4d25dd4fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:42.543324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac26350-a58e-4d39-b839-88a4d25dd4fe None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:42.548819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 554/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:42 2022] GET /identity/v3/role_assignments?user.id=99fbd98cf5b14462a8108692ba32eb19&scope.project.id=2fe586923be749ad968d000e5b733e36&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:44.707306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52da29-cb4c-4a7d-90f4-af48dfc0b36c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:44.707973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52da29-cb4c-4a7d-90f4-af48dfc0b36c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:44.707973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52da29-cb4c-4a7d-90f4-af48dfc0b36c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:44.709302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 564/1118] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:44.715698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad9468-3092-4625-869b-27d7c0c6ea17 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:44.716256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad9468-3092-4625-869b-27d7c0c6ea17 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:44.716256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad9468-3092-4625-869b-27d7c0c6ea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:44.737181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-aaad9468-3092-4625-869b-27d7c0c6ea17 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:44.738424 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aaad9468-3092-4625-869b-27d7c0c6ea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:44.815011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 555/1119] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:45.082646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52da29-cb4c-4a7d-90f4-af48dfc0b36c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:45.187724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52da29-cb4c-4a7d-90f4-af48dfc0b36c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:45.223064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:45.223690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:45.223965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:45.224260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] PATH_INFO: `/v3/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:45.225715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] RBAC: Authorizing `identity:get_role(role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:45.227087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:45.228448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 565/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:45 2022] GET /identity/v3/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:45.234662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad9468-3092-4625-869b-27d7c0c6ea17 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:45.236909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:45.339389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad9468-3092-4625-869b-27d7c0c6ea17 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:45.374956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:45.375592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:45.375870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:45.376135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] PATH_INFO: `/v3/users/99fbd98cf5b14462a8108692ba32eb19` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:45.377345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] RBAC: Authorizing `identity:get_user(user_id=99fbd98cf5b14462a8108692ba32eb19)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:45.378800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:45.380461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 556/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:45 2022] GET /identity/v3/users/99fbd98cf5b14462a8108692ba32eb19 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:45.387006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:45.439652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:45.491442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf513b3-bd07-4295-a852-94a327314b13 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:45.529092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60570156-eea7-4ac5-933c-d445a04ea737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:45.529720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60570156-eea7-4ac5-933c-d445a04ea737 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:45.529988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60570156-eea7-4ac5-933c-d445a04ea737 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:45.530251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60570156-eea7-4ac5-933c-d445a04ea737 None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:45.531374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60570156-eea7-4ac5-933c-d445a04ea737 None admin] RBAC: Authorizing `identity:get_project(project_id=2fe586923be749ad968d000e5b733e36)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:45.532830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60570156-eea7-4ac5-933c-d445a04ea737 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:45.534457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 566/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:49:45 2022] GET /identity/v3/projects/2fe586923be749ad968d000e5b733e36 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:45.541057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:45.612530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:45.646177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747360de-05a9-421c-bd84-8cb32e9eae63 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:45.681646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e0b8d-7a39-4ea7-87ce-91c06615c028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:45.682311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e0b8d-7a39-4ea7-87ce-91c06615c028 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:45.682585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e0b8d-7a39-4ea7-87ce-91c06615c028 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:45.682850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e0b8d-7a39-4ea7-87ce-91c06615c028 None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36/users/99fbd98cf5b14462a8108692ba32eb19/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:45.685396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e0b8d-7a39-4ea7-87ce-91c06615c028 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fe586923be749ad968d000e5b733e36, user_id=99fbd98cf5b14462a8108692ba32eb19, role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:45.686846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e0b8d-7a39-4ea7-87ce-91c06615c028 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:45.701929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 557/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:49:45 2022] PUT /identity/v3/projects/2fe586923be749ad968d000e5b733e36/users/99fbd98cf5b14462a8108692ba32eb19/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:49:47.822803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eccaad-4dbf-4ace-9c67-052023b884da None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:47.823367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eccaad-4dbf-4ace-9c67-052023b884da None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:47.823608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eccaad-4dbf-4ace-9c67-052023b884da None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:47.825036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 567/1124] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:47.832545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f492a5-28ef-4226-aeaf-39a7051b084b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:47.833126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f492a5-28ef-4226-aeaf-39a7051b084b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:47.833126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f492a5-28ef-4226-aeaf-39a7051b084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:47.852933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-07f492a5-28ef-4226-aeaf-39a7051b084b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:47.854141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-07f492a5-28ef-4226-aeaf-39a7051b084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:47.928526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 558/1125] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:48.200319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eccaad-4dbf-4ace-9c67-052023b884da None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:48.306135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eccaad-4dbf-4ace-9c67-052023b884da None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:48.323108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:48.343671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:48.344319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:48.344630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:48.344920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] PATH_INFO: `/v3/roles/8c9bbb1a8df244cea11f2ed1613f2fae` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:48.346375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] RBAC: Authorizing `identity:get_role(role_id=8c9bbb1a8df244cea11f2ed1613f2fae)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:48.347784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:48.349166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 568/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:48 2022] GET /identity/v3/roles/8c9bbb1a8df244cea11f2ed1613f2fae => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:48.355262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f492a5-28ef-4226-aeaf-39a7051b084b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:48.415812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:48.459310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f492a5-28ef-4226-aeaf-39a7051b084b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:48.496738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:48.497376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:48.497654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:48.497921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] PATH_INFO: `/v3/users/99fbd98cf5b14462a8108692ba32eb19` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:48.499109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] RBAC: Authorizing `identity:get_user(user_id=99fbd98cf5b14462a8108692ba32eb19)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:48.500595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:48.502208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 559/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:48 2022] GET /identity/v3/users/99fbd98cf5b14462a8108692ba32eb19 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:48.508464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:48.611654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2929-8a05-4780-bc45-a0d737862c13 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:48.647415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:48.648035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:48.648310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:48.648611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:48.704002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:48.721144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:48.722543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:48.822539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 569/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:49:48.828896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:48.934846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba918-e554-4d2c-81b9-a6c30d1e1378 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:48.963521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:48.972452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c52bfb-623c-4146-9160-18408dd92e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:48.973032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52bfb-623c-4146-9160-18408dd92e45 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:48.973311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52bfb-623c-4146-9160-18408dd92e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:48.973593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52bfb-623c-4146-9160-18408dd92e45 None admin] PATH_INFO: `/v3/projects/2fe586923be749ad968d000e5b733e36` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:48.974801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c52bfb-623c-4146-9160-18408dd92e45 None admin] RBAC: Authorizing `identity:get_project(project_id=2fe586923be749ad968d000e5b733e36)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:48.976258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c52bfb-623c-4146-9160-18408dd92e45 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:48.977948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 560/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:49:48 2022] GET /identity/v3/projects/2fe586923be749ad968d000e5b733e36 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:48.984417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:49.073158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:49.090649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de17a8f-4ecf-485c-ad43-95127d2b389e None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:49.123386 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1f5c1-63a8-4cd9-9022-839b3a532cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:49.123978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f5c1-63a8-4cd9-9022-839b3a532cfd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:49.124253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f5c1-63a8-4cd9-9022-839b3a532cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:49.124544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f5c1-63a8-4cd9-9022-839b3a532cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:49.125304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1f5c1-63a8-4cd9-9022-839b3a532cfd None admin] RBAC: Adding query filter params (role.id=8c9bbb1a8df244cea11f2ed1613f2fae, scope.project.id=2fe586923be749ad968d000e5b733e36, user.id=99fbd98cf5b14462a8108692ba32eb19) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:49.125609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1f5c1-63a8-4cd9-9022-839b3a532cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:49.127037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1f5c1-63a8-4cd9-9022-839b3a532cfd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:49.133088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 570/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:48 2022] GET /identity/v3/role_assignments?user.id=99fbd98cf5b14462a8108692ba32eb19&scope.project.id=2fe586923be749ad968d000e5b733e36&role.id=8c9bbb1a8df244cea11f2ed1613f2fae => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:51.297528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f0f6-1d4f-4a76-a7ff-a74ceb680288 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:51.298141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f0f6-1d4f-4a76-a7ff-a74ceb680288 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:51.298141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f0f6-1d4f-4a76-a7ff-a74ceb680288 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:51.299417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 561/1131] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:51.305433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e9bf9-4c4e-44a1-95db-4733919a6437 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:51.305967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e9bf9-4c4e-44a1-95db-4733919a6437 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:51.305967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e9bf9-4c4e-44a1-95db-4733919a6437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:51.325740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5f9e9bf9-4c4e-44a1-95db-4733919a6437 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:51.326973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5f9e9bf9-4c4e-44a1-95db-4733919a6437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:51.401931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 571/1132] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:51.672887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38f0f6-1d4f-4a76-a7ff-a74ceb680288 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:51.784755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38f0f6-1d4f-4a76-a7ff-a74ceb680288 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:51.820461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de387e1b-3a8d-46a5-bdc5-f0d9016f0722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:51.821112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de387e1b-3a8d-46a5-bdc5-f0d9016f0722 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:51.821382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de387e1b-3a8d-46a5-bdc5-f0d9016f0722 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:51.821651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de387e1b-3a8d-46a5-bdc5-f0d9016f0722 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:51.822824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de387e1b-3a8d-46a5-bdc5-f0d9016f0722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:51.824275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de387e1b-3a8d-46a5-bdc5-f0d9016f0722 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:51.828567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 562/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:51.836965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e9bf9-4c4e-44a1-95db-4733919a6437 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:51.916711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:51.939792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e9bf9-4c4e-44a1-95db-4733919a6437 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:51.974773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbe4b6-a14a-40e3-93c4-7f0551ba8b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:51.975378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbe4b6-a14a-40e3-93c4-7f0551ba8b4a None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:51.975652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbe4b6-a14a-40e3-93c4-7f0551ba8b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:51.975922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbe4b6-a14a-40e3-93c4-7f0551ba8b4a None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:51.976600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbe4b6-a14a-40e3-93c4-7f0551ba8b4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:51.978041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbe4b6-a14a-40e3-93c4-7f0551ba8b4a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:52.006513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 572/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:49:51 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:54.198540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6f7f4-ee10-4dbe-9387-905758678554 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:54.199154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6f7f4-ee10-4dbe-9387-905758678554 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:54.199154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6f7f4-ee10-4dbe-9387-905758678554 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:54.200450 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 563/1135] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:54.206613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f814e-f10a-4db7-9d39-a58f9ba87f8d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:54.207147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f814e-f10a-4db7-9d39-a58f9ba87f8d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:54.207147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f814e-f10a-4db7-9d39-a58f9ba87f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:54.234359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-042f814e-f10a-4db7-9d39-a58f9ba87f8d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:54.236302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-042f814e-f10a-4db7-9d39-a58f9ba87f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:54.317319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 573/1136] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:54.579301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6f7f4-ee10-4dbe-9387-905758678554 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:54.645919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:54.688966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6f7f4-ee10-4dbe-9387-905758678554 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:54.726056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddf2de-670a-4785-b81d-2b7151f72f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:54.726691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddf2de-670a-4785-b81d-2b7151f72f24 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:54.726964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddf2de-670a-4785-b81d-2b7151f72f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:54.727216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddf2de-670a-4785-b81d-2b7151f72f24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:54.728380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddf2de-670a-4785-b81d-2b7151f72f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:54.729867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddf2de-670a-4785-b81d-2b7151f72f24 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:54.731455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 564/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:54.738041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f814e-f10a-4db7-9d39-a58f9ba87f8d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:54.799212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:54.841699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f814e-f10a-4db7-9d39-a58f9ba87f8d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:54.876752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef998bf0-4a25-4fa5-8a92-ed095e566cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:54.877358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef998bf0-4a25-4fa5-8a92-ed095e566cdf None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:54.877640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef998bf0-4a25-4fa5-8a92-ed095e566cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:54.877904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef998bf0-4a25-4fa5-8a92-ed095e566cdf None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:54.878561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef998bf0-4a25-4fa5-8a92-ed095e566cdf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:54.879996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef998bf0-4a25-4fa5-8a92-ed095e566cdf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:54.903029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 574/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:49:54 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:49:57.056876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eef0b-4563-494e-9063-6b7741fc1a35 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:57.057539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eef0b-4563-494e-9063-6b7741fc1a35 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:57.057539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eef0b-4563-494e-9063-6b7741fc1a35 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:57.058800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 565/1139] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:49:57.065042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216e836-3125-4a17-95dc-c836019adce0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:57.065629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216e836-3125-4a17-95dc-c836019adce0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:57.065629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216e836-3125-4a17-95dc-c836019adce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:57.085510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7216e836-3125-4a17-95dc-c836019adce0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:49:57.086735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7216e836-3125-4a17-95dc-c836019adce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:49:57.159642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 575/1140] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:49:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:49:57.424026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eef0b-4563-494e-9063-6b7741fc1a35 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:57.531693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eef0b-4563-494e-9063-6b7741fc1a35 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:57.562090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:57.567451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:57.568298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:57.568561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:57.568878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:57.579095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:49:57.583942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:49:57.588660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:49:57.589797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 566/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:49:57.591508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216e836-3125-4a17-95dc-c836019adce0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:57.643100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:57.704753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216e836-3125-4a17-95dc-c836019adce0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:57.739744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc21186-c044-4d57-b596-527189407187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:57.740394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc21186-c044-4d57-b596-527189407187 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:57.740694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc21186-c044-4d57-b596-527189407187 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:57.740965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc21186-c044-4d57-b596-527189407187 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:57.741630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc21186-c044-4d57-b596-527189407187 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:57.741932 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc21186-c044-4d57-b596-527189407187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:57.743308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc21186-c044-4d57-b596-527189407187 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:57.752264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 576/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:57.758796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:57.865544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054e32e-255f-4e5c-9215-011b87f952cc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:57.900875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:57.901493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:57.901765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:57.902029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] PATH_INFO: `/v3/users/c925ace7ab2541c595b77d3dd166599c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:57.915425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] RBAC: Authorizing `identity:get_user(user_id=c925ace7ab2541c595b77d3dd166599c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:57.916901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:57.918494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 567/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:49:57 2022] GET /identity/v3/users/c925ace7ab2541c595b77d3dd166599c => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:57.924818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc21186-c044-4d57-b596-527189407187 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:58.030679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc21186-c044-4d57-b596-527189407187 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:58.039566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:58.066598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:58.067267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:58.067468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:58.067736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:58.138032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:58.139483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:58.240147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 577/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:49:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:49:58.246524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:58.305478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:58.359282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f999309-6b5c-4dea-ab81-a3c6b9bf0f77 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:58.395553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f69a53-9f8d-4771-bd00-655307d7c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:58.396175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f69a53-9f8d-4771-bd00-655307d7c512 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:58.396487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f69a53-9f8d-4771-bd00-655307d7c512 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:58.396752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f69a53-9f8d-4771-bd00-655307d7c512 None admin] PATH_INFO: `/v3/projects/f641f72e5d4b4f95af2dc0f849757161` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:58.397916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f69a53-9f8d-4771-bd00-655307d7c512 None admin] RBAC: Authorizing `identity:get_project(project_id=f641f72e5d4b4f95af2dc0f849757161)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:58.399363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f69a53-9f8d-4771-bd00-655307d7c512 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:58.401045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 568/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:49:58 2022] GET /identity/v3/projects/f641f72e5d4b4f95af2dc0f849757161 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:49:58.407570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:49:58.418053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:49:58.511309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec439a0-6f01-4f1b-89e5-5fc92e58902c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:49:58.548032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebdbd8-1aca-4e41-9631-79c6e658e548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:49:58.548661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebdbd8-1aca-4e41-9631-79c6e658e548 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:49:58.548938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebdbd8-1aca-4e41-9631-79c6e658e548 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:49:58.549203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebdbd8-1aca-4e41-9631-79c6e658e548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:49:58.549928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebdbd8-1aca-4e41-9631-79c6e658e548 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=f641f72e5d4b4f95af2dc0f849757161, user.id=c925ace7ab2541c595b77d3dd166599c) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:49:58.550236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebdbd8-1aca-4e41-9631-79c6e658e548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:49:58.551660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebdbd8-1aca-4e41-9631-79c6e658e548 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:49:58.557366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 578/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:49:58 2022] GET /identity/v3/role_assignments?user.id=c925ace7ab2541c595b77d3dd166599c&scope.project.id=f641f72e5d4b4f95af2dc0f849757161&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:00.696668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7dad-9821-4c5e-8095-3e1fa6ce598c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:00.697281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7dad-9821-4c5e-8095-3e1fa6ce598c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:00.697281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7dad-9821-4c5e-8095-3e1fa6ce598c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:00.698568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 569/1147] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:00.704916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0db4fd-ed17-4153-a33b-b61c4ebbb926 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:00.705496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0db4fd-ed17-4153-a33b-b61c4ebbb926 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:00.705496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0db4fd-ed17-4153-a33b-b61c4ebbb926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:00.725769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1c0db4fd-ed17-4153-a33b-b61c4ebbb926 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:00.726993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1c0db4fd-ed17-4153-a33b-b61c4ebbb926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:00.804180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 579/1148] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:01.064302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7dad-9821-4c5e-8095-3e1fa6ce598c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:01.179838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7dad-9821-4c5e-8095-3e1fa6ce598c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:01.198653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:01.217047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:01.217674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:01.217946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:01.218217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:01.229133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:50:01.233997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:50:01.235060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:50:01.237329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 570/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:50:01.241533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0db4fd-ed17-4153-a33b-b61c4ebbb926 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:01.243755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:01.344676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0db4fd-ed17-4153-a33b-b61c4ebbb926 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:01.380314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:01.381003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:01.381279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:01.381544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:01.382206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:01.382503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:01.383880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:01.392371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 580/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:50:01.398726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:01.506533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024496c-8b59-4d99-9228-685dfa9f3abb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:01.541801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:01.542423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:01.542697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:01.542969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] PATH_INFO: `/v3/users/c925ace7ab2541c595b77d3dd166599c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:01.544132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] RBAC: Authorizing `identity:get_user(user_id=c925ace7ab2541c595b77d3dd166599c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:01.545624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:01.547228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 571/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:50:01 2022] GET /identity/v3/users/c925ace7ab2541c595b77d3dd166599c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:01.553617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:01.619055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:01.660223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069370e8-b8de-4a6a-a4d8-4398616787f8 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:01.694540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5ac0c-fbe1-4317-888c-94f42a66766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:01.695164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5ac0c-fbe1-4317-888c-94f42a66766f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:01.695439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5ac0c-fbe1-4317-888c-94f42a66766f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:01.695701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5ac0c-fbe1-4317-888c-94f42a66766f None admin] PATH_INFO: `/v3/projects/f641f72e5d4b4f95af2dc0f849757161` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:01.696868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5ac0c-fbe1-4317-888c-94f42a66766f None admin] RBAC: Authorizing `identity:get_project(project_id=f641f72e5d4b4f95af2dc0f849757161)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:01.698289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5ac0c-fbe1-4317-888c-94f42a66766f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:01.699911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 581/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:50:01 2022] GET /identity/v3/projects/f641f72e5d4b4f95af2dc0f849757161 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:01.706320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:01.753384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:01.810238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e99f9-db45-4fc8-8391-5323d6f41d5d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:01.845333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321be7d-a705-4629-95a4-754b0bc4b65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:01.845988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321be7d-a705-4629-95a4-754b0bc4b65e None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:01.846260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321be7d-a705-4629-95a4-754b0bc4b65e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:01.846524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321be7d-a705-4629-95a4-754b0bc4b65e None admin] PATH_INFO: `/v3/projects/f641f72e5d4b4f95af2dc0f849757161/users/c925ace7ab2541c595b77d3dd166599c/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:01.848558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321be7d-a705-4629-95a4-754b0bc4b65e None admin] RBAC: Authorizing `identity:create_grant(project_id=f641f72e5d4b4f95af2dc0f849757161, user_id=c925ace7ab2541c595b77d3dd166599c, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:01.850039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321be7d-a705-4629-95a4-754b0bc4b65e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:01.864650 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 572/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:50:01 2022] PUT /identity/v3/projects/f641f72e5d4b4f95af2dc0f849757161/users/c925ace7ab2541c595b77d3dd166599c/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:50:04.007439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42151d2-ae49-4bba-ace4-85fa01d98d44 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:04.007977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42151d2-ae49-4bba-ace4-85fa01d98d44 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:04.007977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42151d2-ae49-4bba-ace4-85fa01d98d44 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:04.009245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 582/1154] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:04.015637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1ff62-4cd5-47fd-a096-f7af40cca34a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:04.016151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1ff62-4cd5-47fd-a096-f7af40cca34a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:04.016272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1ff62-4cd5-47fd-a096-f7af40cca34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:04.036917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dcf1ff62-4cd5-47fd-a096-f7af40cca34a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:04.038160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dcf1ff62-4cd5-47fd-a096-f7af40cca34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:04.117884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 573/1155] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:04.383720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42151d2-ae49-4bba-ace4-85fa01d98d44 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:04.490828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42151d2-ae49-4bba-ace4-85fa01d98d44 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:04.520732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:04.527208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:04.527844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:04.528121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:04.528376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:04.539393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:50:04.544244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:50:04.545358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:50:04.546354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 583/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:50:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:50:04.551697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1ff62-4cd5-47fd-a096-f7af40cca34a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:04.587631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:04.658593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1ff62-4cd5-47fd-a096-f7af40cca34a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:04.695076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:04.695682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:04.695960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:04.696234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:04.696946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:04.697251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:04.698655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:04.707577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 574/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:50:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:04.714128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:04.820805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4e2ab-bf0d-44c1-a326-00c7d4b4cafa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:04.854480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:04.855099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:04.855372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:04.855638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] PATH_INFO: `/v3/users/c925ace7ab2541c595b77d3dd166599c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:04.856823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] RBAC: Authorizing `identity:get_user(user_id=c925ace7ab2541c595b77d3dd166599c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:04.858287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:04.859850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 584/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:50:04 2022] GET /identity/v3/users/c925ace7ab2541c595b77d3dd166599c => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:04.866113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:04.964891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:04.969687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75b581-a979-42f9-b4fc-68c3cc4d5c2d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:05.004473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:05.005080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:05.005376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:05.005773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:05.074426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:05.075882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:05.176694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:05.181793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 575/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:50:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:50:05.188047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:05.253059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:05.295436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77569129-5dc2-4916-aada-17e32635a074 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:05.329343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2c626-072c-4272-a28f-7a97c2f9d5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:05.329963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2c626-072c-4272-a28f-7a97c2f9d5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:05.330237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2c626-072c-4272-a28f-7a97c2f9d5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:05.330505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2c626-072c-4272-a28f-7a97c2f9d5b8 None admin] PATH_INFO: `/v3/projects/f641f72e5d4b4f95af2dc0f849757161` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:05.331670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2c626-072c-4272-a28f-7a97c2f9d5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=f641f72e5d4b4f95af2dc0f849757161)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:05.333137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2c626-072c-4272-a28f-7a97c2f9d5b8 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:05.334822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 585/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:50:05 2022] GET /identity/v3/projects/f641f72e5d4b4f95af2dc0f849757161 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:05.341415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:05.448689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6d7cd-3345-416d-9dbd-a8c7166b9bfc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:05.484230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d24ec-70d8-42ff-b6b3-bd68ce26c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:05.484858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d24ec-70d8-42ff-b6b3-bd68ce26c865 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:05.485140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d24ec-70d8-42ff-b6b3-bd68ce26c865 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:05.485406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d24ec-70d8-42ff-b6b3-bd68ce26c865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:05.486137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d24ec-70d8-42ff-b6b3-bd68ce26c865 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=f641f72e5d4b4f95af2dc0f849757161, user.id=c925ace7ab2541c595b77d3dd166599c) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:05.486444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d24ec-70d8-42ff-b6b3-bd68ce26c865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:05.487887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d24ec-70d8-42ff-b6b3-bd68ce26c865 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:05.494004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 576/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:50:05 2022] GET /identity/v3/role_assignments?user.id=c925ace7ab2541c595b77d3dd166599c&scope.project.id=f641f72e5d4b4f95af2dc0f849757161&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:07.667399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fa7cc-6772-4e9b-bd04-35c12876d173 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:07.668273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fa7cc-6772-4e9b-bd04-35c12876d173 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:07.668273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fa7cc-6772-4e9b-bd04-35c12876d173 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:07.670382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 586/1162] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:07.679095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445658d-f3fc-40f2-aa09-f7a1c16cc855 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:07.679788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445658d-f3fc-40f2-aa09-f7a1c16cc855 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:07.679788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445658d-f3fc-40f2-aa09-f7a1c16cc855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:07.707048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d445658d-f3fc-40f2-aa09-f7a1c16cc855 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:07.709109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d445658d-f3fc-40f2-aa09-f7a1c16cc855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:07.786736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:07.810856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 577/1163] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:08.084473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fa7cc-6772-4e9b-bd04-35c12876d173 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:08.193226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fa7cc-6772-4e9b-bd04-35c12876d173 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:08.220254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:08.230106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:08.230727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:08.231041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:08.231314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:08.302830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:08.304287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:08.404887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 587/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:50:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:50:08.411297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445658d-f3fc-40f2-aa09-f7a1c16cc855 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:08.513166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445658d-f3fc-40f2-aa09-f7a1c16cc855 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:08.547825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d699d-1ab5-427b-9a3c-7d323f7ed128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:08.548464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d699d-1ab5-427b-9a3c-7d323f7ed128 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:08.548743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d699d-1ab5-427b-9a3c-7d323f7ed128 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:08.549015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d699d-1ab5-427b-9a3c-7d323f7ed128 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:08.555924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d699d-1ab5-427b-9a3c-7d323f7ed128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:08.557408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d699d-1ab5-427b-9a3c-7d323f7ed128 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ec9d699d-1ab5-427b-9a3c-7d323f7ed128 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:50:08.569080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:50:08.570252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:50:08.571115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 578/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:50:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:50:08.576675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:08.604729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:08.679255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9159-0238-4183-9dc6-e8b9abc7b79c None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:08.717540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3122d87-8755-407a-ac76-8c7fc2139f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:08.718154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3122d87-8755-407a-ac76-8c7fc2139f37 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:08.718431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3122d87-8755-407a-ac76-8c7fc2139f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:08.718707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3122d87-8755-407a-ac76-8c7fc2139f37 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:08.719353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3122d87-8755-407a-ac76-8c7fc2139f37 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:08.719651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3122d87-8755-407a-ac76-8c7fc2139f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:08.721062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3122d87-8755-407a-ac76-8c7fc2139f37 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:08.728523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 588/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Tue Sep 13 17:50:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:10.856501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e519c-4760-4a50-8ec5-20e3ccf91ee8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:10.857449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e519c-4760-4a50-8ec5-20e3ccf91ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:10.857449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e519c-4760-4a50-8ec5-20e3ccf91ee8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:10.859003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 579/1167] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:10.864545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5ed2f-5f88-48c8-bef7-05b293127722 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:10.865147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5ed2f-5f88-48c8-bef7-05b293127722 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:10.865147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5ed2f-5f88-48c8-bef7-05b293127722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:10.884418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-19d5ed2f-5f88-48c8-bef7-05b293127722 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:10.885690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-19d5ed2f-5f88-48c8-bef7-05b293127722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:10.954402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 589/1168] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:11.218315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e519c-4760-4a50-8ec5-20e3ccf91ee8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:11.273745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:11.324922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e519c-4760-4a50-8ec5-20e3ccf91ee8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:11.360730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab5e3c-f0b2-4134-89a0-9b099f0bac93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:11.361339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5e3c-f0b2-4134-89a0-9b099f0bac93 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:11.361611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5e3c-f0b2-4134-89a0-9b099f0bac93 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:11.361880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab5e3c-f0b2-4134-89a0-9b099f0bac93 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:11.362550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab5e3c-f0b2-4134-89a0-9b099f0bac93 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:11.363932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab5e3c-f0b2-4134-89a0-9b099f0bac93 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:11.381858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 580/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Tue Sep 13 17:50:11 2022] POST /identity/v3/domains => generated 267 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:50:13.545444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cffacc-539b-4e81-af92-301a614042c4 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:13.546012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cffacc-539b-4e81-af92-301a614042c4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:13.546012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cffacc-539b-4e81-af92-301a614042c4 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:13.547335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 590/1170] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:13.553705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd855424-5c2a-407e-ad64-af6b6570e13b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:13.554305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd855424-5c2a-407e-ad64-af6b6570e13b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:13.554409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd855424-5c2a-407e-ad64-af6b6570e13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:13.578790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dd855424-5c2a-407e-ad64-af6b6570e13b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:13.580535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dd855424-5c2a-407e-ad64-af6b6570e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:13.660382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 581/1171] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:13.927566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cffacc-539b-4e81-af92-301a614042c4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:13.953936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:14.034523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cffacc-539b-4e81-af92-301a614042c4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:14.069835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b72cb-ed49-4ec8-a3b4-5203d869ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:14.070471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b72cb-ed49-4ec8-a3b4-5203d869ff1c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:14.070746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b72cb-ed49-4ec8-a3b4-5203d869ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:14.071019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b72cb-ed49-4ec8-a3b4-5203d869ff1c None admin] PATH_INFO: `/v3/domains/65071c999cd1466f9cdb2db6cf4fe2b0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:14.082226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b72cb-ed49-4ec8-a3b4-5203d869ff1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=65071c999cd1466f9cdb2db6cf4fe2b0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:14.083856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b72cb-ed49-4ec8-a3b4-5203d869ff1c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:14.085784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 591/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Tue Sep 13 17:50:13 2022] GET /identity/v3/domains/65071c999cd1466f9cdb2db6cf4fe2b0 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:14.091940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd855424-5c2a-407e-ad64-af6b6570e13b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:14.138169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:14.200540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd855424-5c2a-407e-ad64-af6b6570e13b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:14.239002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4af01-3b17-4649-bee3-a0f44acaed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:14.239599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4af01-3b17-4649-bee3-a0f44acaed22 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:14.239869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4af01-3b17-4649-bee3-a0f44acaed22 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:14.240140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4af01-3b17-4649-bee3-a0f44acaed22 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:14.240841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4af01-3b17-4649-bee3-a0f44acaed22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:14.242280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e4af01-3b17-4649-bee3-a0f44acaed22 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:14.273121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 582/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:50:14 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:50:16.431058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d11713-e4fa-4524-88ae-8bb79376d13f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:16.431707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d11713-e4fa-4524-88ae-8bb79376d13f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:16.431707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d11713-e4fa-4524-88ae-8bb79376d13f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:16.432933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 592/1174] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:16.439072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b849e-55ec-4158-a2ae-73510492386b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:16.439542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b849e-55ec-4158-a2ae-73510492386b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:16.439635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b849e-55ec-4158-a2ae-73510492386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:16.462036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f22b849e-55ec-4158-a2ae-73510492386b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:16.463525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f22b849e-55ec-4158-a2ae-73510492386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:16.537731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 583/1175] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:16.804184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d11713-e4fa-4524-88ae-8bb79376d13f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:16.888727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:16.910633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d11713-e4fa-4524-88ae-8bb79376d13f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:16.945302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5313e-6ed5-49d1-81ba-1465ba1751f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:16.945930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5313e-6ed5-49d1-81ba-1465ba1751f3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:16.946199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5313e-6ed5-49d1-81ba-1465ba1751f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:16.946464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5313e-6ed5-49d1-81ba-1465ba1751f3 None admin] PATH_INFO: `/v3/domains/65071c999cd1466f9cdb2db6cf4fe2b0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:16.947633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5313e-6ed5-49d1-81ba-1465ba1751f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=65071c999cd1466f9cdb2db6cf4fe2b0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:16.949103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5313e-6ed5-49d1-81ba-1465ba1751f3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:16.950671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 593/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Tue Sep 13 17:50:16 2022] GET /identity/v3/domains/65071c999cd1466f9cdb2db6cf4fe2b0 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:16.956718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b849e-55ec-4158-a2ae-73510492386b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:16.981938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:17.062099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b849e-55ec-4158-a2ae-73510492386b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:17.099434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7055b-d3e1-43f9-9247-3c5e952e6d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:17.100045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7055b-d3e1-43f9-9247-3c5e952e6d20 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:17.100324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7055b-d3e1-43f9-9247-3c5e952e6d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:17.100624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7055b-d3e1-43f9-9247-3c5e952e6d20 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:17.101279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7055b-d3e1-43f9-9247-3c5e952e6d20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:17.102742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7055b-d3e1-43f9-9247-3c5e952e6d20 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:17.124938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 584/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:50:16 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:50:19.312701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a89c4-084c-427f-981c-cf8cd7039219 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:19.313294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a89c4-084c-427f-981c-cf8cd7039219 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:19.313294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a89c4-084c-427f-981c-cf8cd7039219 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:19.315290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 594/1178] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:19.329553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ccdbe-dddb-420f-856a-c12b1f55a218 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:19.331195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ccdbe-dddb-420f-856a-c12b1f55a218 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:19.331195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ccdbe-dddb-420f-856a-c12b1f55a218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:19.352962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8d8ccdbe-dddb-420f-856a-c12b1f55a218 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:19.354556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8d8ccdbe-dddb-420f-856a-c12b1f55a218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:19.432078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 585/1179] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:19.699012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a89c4-084c-427f-981c-cf8cd7039219 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:19.805618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a89c4-084c-427f-981c-cf8cd7039219 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:19.840168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:19.840831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:19.841113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:19.841379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:19.852552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:50:19.857322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:50:19.858372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:50:19.859239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 595/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:50:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:50:19.864824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ccdbe-dddb-420f-856a-c12b1f55a218 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:19.876929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:19.969788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ccdbe-dddb-420f-856a-c12b1f55a218 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:20.006473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:20.007086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:20.007362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:20.007629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:20.008297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:20.008626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:20.010015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:20.018369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 586/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:20.024631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:20.037673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:20.130313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721f822-49b7-4908-aa58-6bf4b537a568 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:20.164114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:20.164770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:20.165047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:20.165314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] PATH_INFO: `/v3/users/20823c28262b49fb9699a0f24b5595bb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:20.178836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] RBAC: Authorizing `identity:get_user(user_id=20823c28262b49fb9699a0f24b5595bb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:20.180277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:20.181894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 596/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:50:20 2022] GET /identity/v3/users/20823c28262b49fb9699a0f24b5595bb => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:20.188215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:20.261019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:20.292782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e179a-7ddb-45b1-8847-be569efecff9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:20.327411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:20.328015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:20.328290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:20.328586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:20.397628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:20.399103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:20.467383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:20.499015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 587/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:50:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:50:20.505418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:20.574300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:20.608574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648fd97-dd2f-4e32-88bb-8249548c45da None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:20.643476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ed483-200e-489f-adbb-bc2cf97648d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:20.644099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ed483-200e-489f-adbb-bc2cf97648d3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:20.644373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ed483-200e-489f-adbb-bc2cf97648d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:20.644672 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ed483-200e-489f-adbb-bc2cf97648d3 None admin] PATH_INFO: `/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:20.645835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ed483-200e-489f-adbb-bc2cf97648d3 None admin] RBAC: Authorizing `identity:get_project(project_id=d1c6cd562cf1409ea1fd58e1b7a514f9)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:20.647301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ed483-200e-489f-adbb-bc2cf97648d3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:20.648956 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 597/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:50:20 2022] GET /identity/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:20.655645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:20.762622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb757672-27e9-424b-b6cf-9d76ed8feffc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:20.799499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce80f65-8005-49d0-92d8-8f740c5ef3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:20.800101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80f65-8005-49d0-92d8-8f740c5ef3cb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:20.800380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80f65-8005-49d0-92d8-8f740c5ef3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:20.800686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80f65-8005-49d0-92d8-8f740c5ef3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:20.801418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80f65-8005-49d0-92d8-8f740c5ef3cb None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=d1c6cd562cf1409ea1fd58e1b7a514f9, user.id=20823c28262b49fb9699a0f24b5595bb) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:20.801727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80f65-8005-49d0-92d8-8f740c5ef3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:20.803209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80f65-8005-49d0-92d8-8f740c5ef3cb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:20.808879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 588/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:50:20 2022] GET /identity/v3/role_assignments?user.id=20823c28262b49fb9699a0f24b5595bb&scope.project.id=d1c6cd562cf1409ea1fd58e1b7a514f9&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:22.897701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f9abe-cd7c-48a9-a17d-8e371c308357 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:22.898320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f9abe-cd7c-48a9-a17d-8e371c308357 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:22.898320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f9abe-cd7c-48a9-a17d-8e371c308357 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:22.899570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 598/1186] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:22.905577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2e1e2-ccb1-43aa-ae5d-cd1f78a1bfad None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:22.906121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2e1e2-ccb1-43aa-ae5d-cd1f78a1bfad None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:22.906121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2e1e2-ccb1-43aa-ae5d-cd1f78a1bfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:22.926337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-93f2e1e2-ccb1-43aa-ae5d-cd1f78a1bfad None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:22.927576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-93f2e1e2-ccb1-43aa-ae5d-cd1f78a1bfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:22.962438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:23.007113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 589/1187] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:23.271387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f9abe-cd7c-48a9-a17d-8e371c308357 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:23.377791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f9abe-cd7c-48a9-a17d-8e371c308357 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:23.419169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:23.419801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:23.420081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:23.420356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:23.423180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:23.432240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:50:23.437086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:50:23.438120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:50:23.439052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 599/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:50:23.444647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2e1e2-ccb1-43aa-ae5d-cd1f78a1bfad None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:23.547840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2e1e2-ccb1-43aa-ae5d-cd1f78a1bfad None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:23.584454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:23.585071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:23.585348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:23.585618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:23.586285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:23.586593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:23.587996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:23.596463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 590/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:23.603318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:23.712146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca58e8-e6f8-48f3-a98f-437839a4d80a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:23.747935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:23.748598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:23.748875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:23.749150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] PATH_INFO: `/v3/users/20823c28262b49fb9699a0f24b5595bb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:23.750320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] RBAC: Authorizing `identity:get_user(user_id=20823c28262b49fb9699a0f24b5595bb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:23.751772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:23.753393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 600/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:50:23 2022] GET /identity/v3/users/20823c28262b49fb9699a0f24b5595bb => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:23.759672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:23.790575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:23.860398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f1840-fdd7-4fe5-b669-06e5000880e7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:23.895220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff2b4e-6f5d-4a94-9c96-6967d39cfe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:23.895841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff2b4e-6f5d-4a94-9c96-6967d39cfe0f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:23.896143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff2b4e-6f5d-4a94-9c96-6967d39cfe0f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:23.896415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff2b4e-6f5d-4a94-9c96-6967d39cfe0f None admin] PATH_INFO: `/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:23.897613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff2b4e-6f5d-4a94-9c96-6967d39cfe0f None admin] RBAC: Authorizing `identity:get_project(project_id=d1c6cd562cf1409ea1fd58e1b7a514f9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:23.899058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff2b4e-6f5d-4a94-9c96-6967d39cfe0f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:23.900741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 591/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:50:23 2022] GET /identity/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:23.906975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:23.971280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:24.010735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063ff87-2863-43ca-af04-720b6c9203b1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:24.046508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128f189-94a5-4b37-84bb-552c06203669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:24.047167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128f189-94a5-4b37-84bb-552c06203669 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:24.047445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128f189-94a5-4b37-84bb-552c06203669 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:24.047712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128f189-94a5-4b37-84bb-552c06203669 None admin] PATH_INFO: `/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9/users/20823c28262b49fb9699a0f24b5595bb/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:24.049734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128f189-94a5-4b37-84bb-552c06203669 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1c6cd562cf1409ea1fd58e1b7a514f9, user_id=20823c28262b49fb9699a0f24b5595bb, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:24.051179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128f189-94a5-4b37-84bb-552c06203669 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:24.065316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 601/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:50:23 2022] PUT /identity/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9/users/20823c28262b49fb9699a0f24b5595bb/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 17:50:26.160096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf9011-78ca-4e67-900c-5cfcb326bd6d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:26.160709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf9011-78ca-4e67-900c-5cfcb326bd6d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:26.160709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf9011-78ca-4e67-900c-5cfcb326bd6d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:26.162144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 592/1193] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:50:26.168376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b2a68-db18-4893-bd25-c9843606988e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:26.168944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b2a68-db18-4893-bd25-c9843606988e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:26.168944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b2a68-db18-4893-bd25-c9843606988e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:26.188691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d39b2a68-db18-4893-bd25-c9843606988e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:50:26.189944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d39b2a68-db18-4893-bd25-c9843606988e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:50:26.270276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 602/1194] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:50:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:50:26.541350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf9011-78ca-4e67-900c-5cfcb326bd6d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:26.641351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:26.648887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf9011-78ca-4e67-900c-5cfcb326bd6d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:26.684203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:26.684860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:26.685136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:26.685403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:26.696655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:50:26.701439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:50:26.702492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:50:26.703427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 593/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:50:26.708884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b2a68-db18-4893-bd25-c9843606988e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:26.767769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:26.818725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b2a68-db18-4893-bd25-c9843606988e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:26.854402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:26.855014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:26.855291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:26.855562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:26.856232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:26.856562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:26.857944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:26.866242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 603/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:26.872596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:26.978791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95bef3-08b1-4aa0-a728-5ca89431d61f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:27.014553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:27.015177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:27.015456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:27.015725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] PATH_INFO: `/v3/users/20823c28262b49fb9699a0f24b5595bb` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:27.016891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] RBAC: Authorizing `identity:get_user(user_id=20823c28262b49fb9699a0f24b5595bb)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:27.018349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:27.019960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 594/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Tue Sep 13 17:50:26 2022] GET /identity/v3/users/20823c28262b49fb9699a0f24b5595bb => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:27.026152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:27.124480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9abf38-87ce-424b-a0ed-91d77b5d7562 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:27.142124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:27.162218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:27.162830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:27.163105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:27.163371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:27.230671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:27.232123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:27.332184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 604/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:50:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:50:27.338708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:27.365294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:27.443164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a92429-19fa-413d-8558-d0256e76d79a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:27.477440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2f5e4-9eed-4926-ab00-b8ab38f0588f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:27.477962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2f5e4-9eed-4926-ab00-b8ab38f0588f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:27.478234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2f5e4-9eed-4926-ab00-b8ab38f0588f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:27.478499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2f5e4-9eed-4926-ab00-b8ab38f0588f None admin] PATH_INFO: `/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:27.479653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2f5e4-9eed-4926-ab00-b8ab38f0588f None admin] RBAC: Authorizing `identity:get_project(project_id=d1c6cd562cf1409ea1fd58e1b7a514f9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:27.481112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2f5e4-9eed-4926-ab00-b8ab38f0588f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:27.482737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 595/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Tue Sep 13 17:50:27 2022] GET /identity/v3/projects/d1c6cd562cf1409ea1fd58e1b7a514f9 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:27.489363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:50:27.507594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:27.589431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b89ab-0164-4367-8b64-57c5d12429c7 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:50:27.624008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04c3d4-3885-47cc-a6db-da93f8502847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:50:27.624632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04c3d4-3885-47cc-a6db-da93f8502847 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:50:27.624905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04c3d4-3885-47cc-a6db-da93f8502847 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:50:27.625172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04c3d4-3885-47cc-a6db-da93f8502847 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:50:27.625889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04c3d4-3885-47cc-a6db-da93f8502847 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=d1c6cd562cf1409ea1fd58e1b7a514f9, user.id=20823c28262b49fb9699a0f24b5595bb) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:50:27.626196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04c3d4-3885-47cc-a6db-da93f8502847 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:50:27.627622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04c3d4-3885-47cc-a6db-da93f8502847 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:50:27.633679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 605/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:50:27 2022] GET /identity/v3/role_assignments?user.id=20823c28262b49fb9699a0f24b5595bb&scope.project.id=d1c6cd562cf1409ea1fd58e1b7a514f9&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:50:37.472448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=54 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:50:37.630823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=60 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:25.520207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6859b-422b-4060-bebd-a1b628f994ac None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:25.521164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6859b-422b-4060-bebd-a1b628f994ac None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:25.521164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6859b-422b-4060-bebd-a1b628f994ac None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:25.523791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 596/1201] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:51:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:51:25.532217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e58570-e4b8-4eaa-8dda-a0580b9b0a74 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:25.533048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e58570-e4b8-4eaa-8dda-a0580b9b0a74 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:25.533048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e58570-e4b8-4eaa-8dda-a0580b9b0a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:25.558634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b2e58570-e4b8-4eaa-8dda-a0580b9b0a74 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:51:25.560276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b2e58570-e4b8-4eaa-8dda-a0580b9b0a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:51:25.707187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 606/1202] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:51:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:51:26.188289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6859b-422b-4060-bebd-a1b628f994ac None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:26.413652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6859b-422b-4060-bebd-a1b628f994ac None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:26.490497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:26.491386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:26.491856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:26.492269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:26.516255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:26.517924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:26.541744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:26.549644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:26.550824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:26.550824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:26.550824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:26.550824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 597/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:51:26.552869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e58570-e4b8-4eaa-8dda-a0580b9b0a74 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:26.751270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e58570-e4b8-4eaa-8dda-a0580b9b0a74 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:26.796775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:26.809422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96be91-5119-47f0-b9f6-bd3c0ecba69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:26.810442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96be91-5119-47f0-b9f6-bd3c0ecba69f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:26.810929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96be91-5119-47f0-b9f6-bd3c0ecba69f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:26.811406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96be91-5119-47f0-b9f6-bd3c0ecba69f None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:26.813359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96be91-5119-47f0-b9f6-bd3c0ecba69f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:26.813912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96be91-5119-47f0-b9f6-bd3c0ecba69f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:26.816298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96be91-5119-47f0-b9f6-bd3c0ecba69f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:26.836495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 607/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:26.844845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:26.961831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:27.091859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397c9a-46d7-4723-9029-dff373364420 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:27.141322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54bd32-e0ca-48f6-94e7-02fa4d85be94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:27.142285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54bd32-e0ca-48f6-94e7-02fa4d85be94 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:27.142551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54bd32-e0ca-48f6-94e7-02fa4d85be94 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:27.142826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54bd32-e0ca-48f6-94e7-02fa4d85be94 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:27.143477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54bd32-e0ca-48f6-94e7-02fa4d85be94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:27.145442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54bd32-e0ca-48f6-94e7-02fa4d85be94 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:27.196523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 598/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Tue Sep 13 17:51:26 2022] POST /identity/v3/users => generated 251 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 17:51:29.917102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe367a38-50a0-4dde-8ad2-01d620754b6c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:29.918200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe367a38-50a0-4dde-8ad2-01d620754b6c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:29.918200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe367a38-50a0-4dde-8ad2-01d620754b6c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:29.924798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 608/1206] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:51:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:51:29.931837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb451d58-b16e-47bd-b15e-a23a14b6d780 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:29.932409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb451d58-b16e-47bd-b15e-a23a14b6d780 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:29.932409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb451d58-b16e-47bd-b15e-a23a14b6d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:29.955989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bb451d58-b16e-47bd-b15e-a23a14b6d780 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:51:29.957832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb451d58-b16e-47bd-b15e-a23a14b6d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:51:30.074968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 599/1207] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:51:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:51:30.515230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe367a38-50a0-4dde-8ad2-01d620754b6c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:30.671014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe367a38-50a0-4dde-8ad2-01d620754b6c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:30.732794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:30.734194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:30.734593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:30.735594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:30.750971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:51:30.758392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:30.761362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:30.763619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 609/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:51:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:51:30.768043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb451d58-b16e-47bd-b15e-a23a14b6d780 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:30.833356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:30.931420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb451d58-b16e-47bd-b15e-a23a14b6d780 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:30.994086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:30.995137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:30.995615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:30.996091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:30.997258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:30.997724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:30.999934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:31.024946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 600/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:51:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:31.027003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:31.116072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:31.214161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca9d8f-2cf2-40bb-aacc-1711c9ea1da9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:31.265104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:31.265769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:31.266081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:31.266318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:31.279680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:31.281147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:31.298413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:31.300460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:31.302336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:31.302336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:31.302336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:31.302336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 610/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:51:31.307880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:31.492573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:31.498616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c908197-3d24-4dc4-85a2-2f38c4b9c1c9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:31.567710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:31.568831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:31.569335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:31.569824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:31.571040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:31.571587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:31.574290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:31.604234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 601/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:51:31.606334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:31.811842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b50fe-f035-464c-ba12-bd2934064653 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:31.815188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:31.891138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:31.892180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:31.892762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:31.893254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:31.910169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:31.913157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:51:31.943656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:31.945859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:31.947862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:31.956864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 611/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:31.957048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:32.158884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c797f-06f0-4999-9696-2a45bf6e0548 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:32.225489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:32.226499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:32.226967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:32.227491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:32.229096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:32.229619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:32.232361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:32.275395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 602/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 13 17:51:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:32.277642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:32.461990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa3fb2-d8e5-45a0-9b41-c43ab001f19d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:32.509059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:32.509809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:32.510119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:32.510494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:32.549883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:32.595003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:32.596640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:32.730544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 612/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:51:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:51:32.738111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:32.763222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:32.884852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f886f9a-5e18-45e0-9ffc-1bf667887621 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:32.928447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:32.929089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:32.929369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:32.929660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:32.941261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:32.942893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:32.959334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:32.961275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:32.963032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:32.963032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:32.963032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:32.963032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 603/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:51:32.967216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:33.051813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:33.105265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15790084-7e27-45bf-a75d-b12824a9289f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:33.159113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:33.159960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:33.160461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:33.160807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:33.161660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:33.162068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:33.163708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:33.179753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 613/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:33.186079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:33.290752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:33.347247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78043e-afb5-4f21-a3d5-f05afcdb207b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:33.407618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:33.408871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:33.409300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:33.409614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:33.422107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:33.424918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:51:33.450907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:33.452280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:33.453644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 604/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:33.459789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:33.589927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4064-b7af-46e3-bafe-7a9c55b7176d None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:33.623885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:33.641598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfa9cb-3d6d-461c-a17b-45714325ecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:33.642642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa9cb-3d6d-461c-a17b-45714325ecdf None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:33.643092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa9cb-3d6d-461c-a17b-45714325ecdf None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:33.643446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfa9cb-3d6d-461c-a17b-45714325ecdf None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:33.644742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfa9cb-3d6d-461c-a17b-45714325ecdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:33.645202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfa9cb-3d6d-461c-a17b-45714325ecdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:33.647201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfa9cb-3d6d-461c-a17b-45714325ecdf None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:33.667307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 614/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:33.676552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:33.846251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9d8bc-c3c6-4abc-98ad-17772b744e08 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:33.884064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:33.895051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f35b0-7cd4-4c39-9a27-1da15bcd7fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:33.895799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f35b0-7cd4-4c39-9a27-1da15bcd7fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:33.896087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f35b0-7cd4-4c39-9a27-1da15bcd7fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:33.896591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f35b0-7cd4-4c39-9a27-1da15bcd7fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:33.897646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f35b0-7cd4-4c39-9a27-1da15bcd7fa0 None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=c6ce465c1ee84703b7b44d33a91d7017) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:33.897991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f35b0-7cd4-4c39-9a27-1da15bcd7fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:33.899764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f35b0-7cd4-4c39-9a27-1da15bcd7fa0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:33.910062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 605/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:51:33 2022] GET /identity/v3/role_assignments?user.id=c6ce465c1ee84703b7b44d33a91d7017&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:36.379545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7937-35da-4af9-8251-f1c39aa63c0a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:36.380219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7937-35da-4af9-8251-f1c39aa63c0a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:36.380219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7937-35da-4af9-8251-f1c39aa63c0a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:36.381565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 615/1220] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:51:36.387635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759be082-9e37-418e-9173-bb36dc78f878 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:36.388206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759be082-9e37-418e-9173-bb36dc78f878 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:36.388206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759be082-9e37-418e-9173-bb36dc78f878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:36.408605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-759be082-9e37-418e-9173-bb36dc78f878 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:51:36.409925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-759be082-9e37-418e-9173-bb36dc78f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:51:36.500197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 606/1221] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:51:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:51:36.810573 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c7937-35da-4af9-8251-f1c39aa63c0a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:36.953246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c7937-35da-4af9-8251-f1c39aa63c0a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:36.998905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:36.999469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:36.999751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:37.000068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:37.012655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:51:37.019276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:37.020820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:37.022256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 616/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:51:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:37.027849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759be082-9e37-418e-9173-bb36dc78f878 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:37.159890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759be082-9e37-418e-9173-bb36dc78f878 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:37.192029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:37.205925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:37.206638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:37.206918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:37.207251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:37.207911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:37.208255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:37.210123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:37.219768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 607/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:51:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:37.226599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:37.281007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:37.348169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301bd5b-b8fe-4af4-a7bc-b29bad0ece98 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:37.387199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:37.387907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:37.388152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:37.388484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:37.400998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:37.402647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:37.419958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:37.421307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:37.422418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:37.422418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:37.422418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:37.422418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 617/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:37.427961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:37.557680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6b96-a972-4170-b5b6-223a2244b67f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:37.600343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:37.601033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:37.601332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:37.601726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:37.602392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:37.602720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:37.604262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:37.620004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 608/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:37.627436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:37.749878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:37.758709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb5682-70a1-493f-97fd-dd5480f899aa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:37.803182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:37.803814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:37.804125 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:37.804448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:37.812809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:37.814211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:51:37.826598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:37.827669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:37.828659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 618/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:51:37.834666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:37.895007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:37.970699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece38342-68f2-4f35-868c-1cc291240c2e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:38.015374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:38.017207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:38.017530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:38.017792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:38.018760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:38.019060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:38.020697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:38.044712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 609/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 13 17:51:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:51:38.051147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:38.184482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01378f24-926b-40c5-99fa-8fdcbea2b504 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:38.221734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:38.222384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:38.222662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:38.222931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:38.242073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:38.244174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:38.268197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:38.269634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:38.270979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:38.270979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:38.270979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:38.272455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 619/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:38.279739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:38.411019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a86fd6-2094-43ad-bd48-2f0da5e4c721 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:38.413750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:38.451754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:38.452452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:38.452668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:38.452983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:38.453845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:38.454215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:38.455691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:38.479880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 610/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:38.482986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:38.496855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:38.616662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6627876-5ec4-44c1-9a41-fd5776530632 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:38.668649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:38.669875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:38.670354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:38.670684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:38.681857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:38.683503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:51:38.705089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:38.706221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:38.707367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 620/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:38.715597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:38.839107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddeb220-9315-4fca-80cc-5072828b6ad3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:38.884549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8589fd-71d7-4ec1-9525-31ac3489dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:38.885246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8589fd-71d7-4ec1-9525-31ac3489dcdc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:38.885545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8589fd-71d7-4ec1-9525-31ac3489dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:38.885875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8589fd-71d7-4ec1-9525-31ac3489dcdc None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:38.886692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8589fd-71d7-4ec1-9525-31ac3489dcdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:38.887407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8589fd-71d7-4ec1-9525-31ac3489dcdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:38.889204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8589fd-71d7-4ec1-9525-31ac3489dcdc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:38.916138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 611/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:38.922070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:38.999724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:39.050775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f694d7e-14e2-4f65-9f0f-4056e813ac52 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:39.106397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95839db5-2707-4e13-8ee9-017350ce9c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:39.107765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95839db5-2707-4e13-8ee9-017350ce9c18 None admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:39.108350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95839db5-2707-4e13-8ee9-017350ce9c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:39.109498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95839db5-2707-4e13-8ee9-017350ce9c18 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/c6ce465c1ee84703b7b44d33a91d7017/roles/cb2c16a6113142179869b36ff210c590` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:39.129211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95839db5-2707-4e13-8ee9-017350ce9c18 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=c6ce465c1ee84703b7b44d33a91d7017, role_id=cb2c16a6113142179869b36ff210c590)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:39.130734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95839db5-2707-4e13-8ee9-017350ce9c18 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:39.150127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 621/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:51:38 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/c6ce465c1ee84703b7b44d33a91d7017/roles/cb2c16a6113142179869b36ff210c590 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:51:41.644329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa254a-fdf0-4ee9-945b-50d6e56f8994 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:41.645079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa254a-fdf0-4ee9-945b-50d6e56f8994 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:41.645079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa254a-fdf0-4ee9-945b-50d6e56f8994 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:41.646851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 612/1233] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:51:41.654469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311928f4-c1d5-4a92-8763-fc63a7e52a85 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:41.655397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311928f4-c1d5-4a92-8763-fc63a7e52a85 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:41.655397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311928f4-c1d5-4a92-8763-fc63a7e52a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:41.675562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-311928f4-c1d5-4a92-8763-fc63a7e52a85 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:51:41.676892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-311928f4-c1d5-4a92-8763-fc63a7e52a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:51:41.773817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 622/1234] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:51:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 17:51:42.077161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa254a-fdf0-4ee9-945b-50d6e56f8994 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:42.112667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:42.225506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa254a-fdf0-4ee9-945b-50d6e56f8994 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:42.270447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:42.271218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:42.271595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:42.271947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:42.285300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:51:42.291639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:42.292826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:42.293681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 613/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Tue Sep 13 17:51:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:42.302104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311928f4-c1d5-4a92-8763-fc63a7e52a85 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:42.367838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:42.446079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311928f4-c1d5-4a92-8763-fc63a7e52a85 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:42.501328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:42.501952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:42.502257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:42.502718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:42.503343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:42.503646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:42.505255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:42.515922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 623/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Tue Sep 13 17:51:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:42.522616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:42.631227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:42.649926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1124-d827-4181-9348-ea66b3278337 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:42.692971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:42.693702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:42.693986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:42.694258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:42.709715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:42.711550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:42.729311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:42.730472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:42.731449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:42.731449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:42.731449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:42.731449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 614/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:42.742400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:42.865314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdbd25-0574-4880-97c9-d545022fbad6 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:42.881501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:42.919269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:42.920112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:42.920347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:42.920705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:42.921401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:42.921776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:42.923559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:42.939214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 624/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:42.948225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:43.069438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1a8a9-d139-4983-802f-61131655ac64 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:43.109882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:43.110572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:43.110919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:43.111239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:43.115139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:43.122683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:43.124667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:51:43.140373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:43.142077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:43.146663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 615/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:43.148679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:43.288327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4434ca5-1659-42df-bc11-2a780a66a483 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:43.335409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:43.336175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:43.336515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:43.336830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:43.337730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:43.337959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:43.342945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:43.360799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 625/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 13 17:51:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:43.368644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:43.515110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1dd3-4dd0-460e-870e-979246158ce9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:43.559760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:43.560476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:43.560876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:43.561193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:43.659740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:43.668974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:43.670623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:43.796910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 616/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:51:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:51:43.803174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:43.842021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:43.925738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d8a29-3522-4016-99af-ce48c78836b0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:43.970508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:43.971238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:43.971617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:43.972018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:43.990580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:43.993284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:44.018780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:44.020266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:44.027784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:44.027784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:44.027784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:44.027784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 626/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:44.030430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:44.181252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:44.190531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6624-eb8e-452e-a4fa-f3c384c4e4aa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:44.242038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:44.242812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:44.243152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:44.243490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:44.244491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:44.244989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:44.247198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:44.266729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 617/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:44.273566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:44.380214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:44.407200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7b54d-3188-4c07-a1c6-47ec915e7795 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:44.449974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:44.450517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:44.450852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:44.451161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:44.459979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:44.461710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:51:44.475739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:44.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:44.479246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 627/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:51:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:44.485160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:44.648486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4b673-e5e3-4a46-a198-82ef8baa6eba None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:44.707294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7a280-1c35-43eb-9ff9-fb33a63369ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:44.707968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a280-1c35-43eb-9ff9-fb33a63369ca None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:44.708184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a280-1c35-43eb-9ff9-fb33a63369ca None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:44.708485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a280-1c35-43eb-9ff9-fb33a63369ca None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:44.709353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a280-1c35-43eb-9ff9-fb33a63369ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:44.709698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a280-1c35-43eb-9ff9-fb33a63369ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:44.711412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a280-1c35-43eb-9ff9-fb33a63369ca None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:44.739514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 618/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:51:44.741489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:44.899559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d3f9d-421f-4c3f-a903-1238caa3cfc9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:44.950472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb5bce-7633-4aee-8faf-1c3f7cd2a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:44.951778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb5bce-7633-4aee-8faf-1c3f7cd2a52c None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:44.951971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb5bce-7633-4aee-8faf-1c3f7cd2a52c None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:44.952245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb5bce-7633-4aee-8faf-1c3f7cd2a52c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:44.953294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb5bce-7633-4aee-8faf-1c3f7cd2a52c None admin] RBAC: Adding query filter params (role.id=cb2c16a6113142179869b36ff210c590, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=c6ce465c1ee84703b7b44d33a91d7017) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:44.953783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb5bce-7633-4aee-8faf-1c3f7cd2a52c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:44.955920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb5bce-7633-4aee-8faf-1c3f7cd2a52c None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:44.965370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 628/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:51:44 2022] GET /identity/v3/role_assignments?user.id=c6ce465c1ee84703b7b44d33a91d7017&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=cb2c16a6113142179869b36ff210c590 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:47.570458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de031-31f3-4a32-b5b2-9a59452454ec None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:47.571309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de031-31f3-4a32-b5b2-9a59452454ec None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:47.571309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de031-31f3-4a32-b5b2-9a59452454ec None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:47.572645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 619/1247] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:51:47.579572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e86c1-20f5-4280-be43-11ca2b543f9f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:47.580447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e86c1-20f5-4280-be43-11ca2b543f9f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:47.580603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e86c1-20f5-4280-be43-11ca2b543f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:47.588183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:47.602628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-197e86c1-20f5-4280-be43-11ca2b543f9f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:51:47.603969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-197e86c1-20f5-4280-be43-11ca2b543f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:51:47.711382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 629/1248] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:51:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:51:48.059239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1de031-31f3-4a32-b5b2-9a59452454ec None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:48.086930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:48.226215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1de031-31f3-4a32-b5b2-9a59452454ec None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:48.273263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:48.274072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:48.274350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:48.274659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:48.288728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:51:48.294733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:48.296183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:48.297665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 620/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:48.303126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e86c1-20f5-4280-be43-11ca2b543f9f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:48.431303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e86c1-20f5-4280-be43-11ca2b543f9f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:48.475767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:48.484344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:48.485486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:48.485816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:48.486125 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:48.486812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:48.487169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:48.488763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:48.501553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 630/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:51:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:48.509634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:48.619592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:48.652035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdff50-c8f9-489f-94d1-16cc4cc9a01a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:48.705525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:48.706362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:48.706698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:48.706976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:48.724244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:48.726827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:48.749495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:48.750568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:48.757274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:48.757274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:48.757274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:48.757274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 621/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:48.758051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:48.906940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6e3b-359d-4052-ac40-49734acbd968 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:48.951180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:48.951831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:48.952124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:48.952392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:48.953121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:48.953449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:48.955002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:48.969288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 631/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:48.975474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:49.113107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ef90a-e0fc-48a5-ab04-30003eb59267 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:49.153357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:49.163762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:49.164477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:49.164836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:49.165089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:49.175040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:49.176639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:51:49.190095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:49.191937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:49.193629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 622/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 17:51:49.198579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:49.279647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:49.334829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47aaa2-d23f-4570-bda5-1ea99838564a None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:49.380798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:49.381453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:49.381735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:49.382038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:49.382861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:49.383167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:49.384978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:49.409999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 632/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 13 17:51:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:49.418353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:49.558662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ba7aa-a74b-4ead-9c7a-0bd20f464538 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:49.614065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:49.614723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:49.615126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:49.615396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:49.697869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:49.716953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:49.718477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:49.870786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 623/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:51:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:51:49.877188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:49.999914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9959943-1187-434b-bec8-cc2cba8c38fe None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:50.017212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:50.043043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:50.043699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:50.043980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:50.044255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:50.055884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:50.057413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:50.075745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:50.076983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:50.078145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:50.078145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:50.078145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:50.080951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 633/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:50.083256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:50.207698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:50.221591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a5caa-2300-4b9b-926b-55a3640fb4d3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:50.259989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:50.260642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:50.260918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:50.261189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:50.261848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:50.262508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:50.263924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:50.277097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 624/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:50.283146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:50.400086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378e0b8-3c0a-4724-8ea7-e575a43dc459 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:50.443153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:50.443943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:50.444259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:50.444558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:50.452080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:50.453638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:51:50.467570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:50.468712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:50.469754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 634/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:51:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:50.475545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:50.611047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1493e-57e0-4808-8182-7cc86cecf450 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:50.649888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90090db1-c606-4385-bff8-993e38618cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:50.650533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90090db1-c606-4385-bff8-993e38618cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:50.650807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90090db1-c606-4385-bff8-993e38618cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:50.651075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90090db1-c606-4385-bff8-993e38618cc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:50.651777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90090db1-c606-4385-bff8-993e38618cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:50.652073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90090db1-c606-4385-bff8-993e38618cc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:50.653560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90090db1-c606-4385-bff8-993e38618cc2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:50.667328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 625/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:51:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:50.675002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:50.707462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:50.804825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb86c2-4570-4a96-807f-777ee33b09dd None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:50.846522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d075ed-c2cc-4388-a874-5b8fe98b2149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:50.847146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d075ed-c2cc-4388-a874-5b8fe98b2149 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:50.847422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d075ed-c2cc-4388-a874-5b8fe98b2149 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:50.847695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d075ed-c2cc-4388-a874-5b8fe98b2149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:50.848412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d075ed-c2cc-4388-a874-5b8fe98b2149 None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=c6ce465c1ee84703b7b44d33a91d7017) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:50.848740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d075ed-c2cc-4388-a874-5b8fe98b2149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:50.850251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d075ed-c2cc-4388-a874-5b8fe98b2149 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:50.856016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 635/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:51:50 2022] GET /identity/v3/role_assignments?user.id=c6ce465c1ee84703b7b44d33a91d7017&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:53.062135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b58c6-6a01-4a14-9506-559c8e9320cb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:53.062695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b58c6-6a01-4a14-9506-559c8e9320cb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:53.062695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b58c6-6a01-4a14-9506-559c8e9320cb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:53.064057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 626/1261] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:51:53.069739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832862b8-a686-4039-ae06-25945dd1e519 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:53.070165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832862b8-a686-4039-ae06-25945dd1e519 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:53.070275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832862b8-a686-4039-ae06-25945dd1e519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:53.089793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-832862b8-a686-4039-ae06-25945dd1e519 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:51:53.091010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-832862b8-a686-4039-ae06-25945dd1e519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:51:53.181980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 636/1262] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:51:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:51:53.447662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b58c6-6a01-4a14-9506-559c8e9320cb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:53.458949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:53.554953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b58c6-6a01-4a14-9506-559c8e9320cb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:53.590046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:53.590663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:53.590939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:53.591204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:53.601848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:51:53.606633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:53.607718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:53.608991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 627/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:51:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:53.613930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832862b8-a686-4039-ae06-25945dd1e519 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:53.633561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:53.719364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832862b8-a686-4039-ae06-25945dd1e519 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:53.756095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:53.756741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:53.757020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:53.757294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:53.758032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:53.758341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:53.759723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:53.768443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 637/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:51:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:53.774513 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:53.844942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:53.880112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a335a-e2e0-4bcb-9052-99a441d18455 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:53.917749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:53.918377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:53.918650 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:53.918917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:53.929509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:53.930964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:53.945864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:53.947036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:53.948388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:53.948388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:53.948388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:53.948388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 628/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:53.952971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:54.038873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:54.058852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9dcd8-cac9-4c37-9bfb-472f7b6ede29 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:54.093989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:54.094680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:54.094864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:54.095133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:54.095873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:54.096247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:54.097652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:54.111054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 638/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:54.118019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:54.231381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fce32-cb39-4c7c-a4e0-9dc37ca53446 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:54.251684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:54.269421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:54.270079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:54.270357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:54.270622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:54.280025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:54.281534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:51:54.295434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:54.296595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:54.297950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 629/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:54.302745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:54.421925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac91e8-e9e4-4391-bff4-028976db723f None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:54.458630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:54.459242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:54.459519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:54.459791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:54.460545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:54.460851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:54.462295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:54.471544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:54.481722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 639/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 13 17:51:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:54.488127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:54.602874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363c58f-3659-4861-a302-5d7ea151c4af None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:54.637855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:54.638476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:54.638724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:54.639025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:54.649838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:54.651395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:54.672312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:54.673479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:54.674579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:54.674579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:54.674579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:54.676775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 630/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:54.679585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:54.791279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617daad-c008-4382-a37a-c93b24c4d124 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:54.829407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:54.830019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:54.830305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:54.830596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:54.831221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:54.831524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:54.832956 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:54.844635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 640/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:54.850699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:54.883346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:54.953413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32b5fd-e62d-4018-b49a-4963f4b38c5f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:54.988926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:54.989553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:54.989829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:54.990188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:54.997126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:54.998588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:51:55.010097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:55.011181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:55.012752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 631/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:51:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:55.017665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:55.081675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:55.133047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fc722-f003-4dff-b9a2-f7c7188545ed None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:55.168943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d89076-9f3b-47ce-937d-115c75088c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:55.169461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89076-9f3b-47ce-937d-115c75088c43 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:55.169734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89076-9f3b-47ce-937d-115c75088c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:55.169998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d89076-9f3b-47ce-937d-115c75088c43 None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:55.170689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89076-9f3b-47ce-937d-115c75088c43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:55.170989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89076-9f3b-47ce-937d-115c75088c43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:55.172416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d89076-9f3b-47ce-937d-115c75088c43 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:55.185235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 641/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:55.191605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:55.294412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:55.304321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd60cb0-aa70-4a0b-a7b1-03f04ccf9dd2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:55.340291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79553-405a-44e3-84f3-aae06337b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:55.340973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79553-405a-44e3-84f3-aae06337b2b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:55.341246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79553-405a-44e3-84f3-aae06337b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:55.341521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79553-405a-44e3-84f3-aae06337b2b9 None admin] PATH_INFO: `/v3/projects/2b736c723c894226850e560a2dd28985/users/c6ce465c1ee84703b7b44d33a91d7017/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:55.343526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79553-405a-44e3-84f3-aae06337b2b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b736c723c894226850e560a2dd28985, user_id=c6ce465c1ee84703b7b44d33a91d7017, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:55.345006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79553-405a-44e3-84f3-aae06337b2b9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:55.360657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 632/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Tue Sep 13 17:51:55 2022] PUT /identity/v3/projects/2b736c723c894226850e560a2dd28985/users/c6ce465c1ee84703b7b44d33a91d7017/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:51:57.519634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4c63-9a16-4717-8615-6f545e857f5c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:57.520205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4c63-9a16-4717-8615-6f545e857f5c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:57.520205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4c63-9a16-4717-8615-6f545e857f5c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:57.521609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 642/1274] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:51:57.526988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e407bc-47a5-4c05-91a1-1db40e6c0e76 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:57.527414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e407bc-47a5-4c05-91a1-1db40e6c0e76 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:57.527504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e407bc-47a5-4c05-91a1-1db40e6c0e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:57.548692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a9e407bc-47a5-4c05-91a1-1db40e6c0e76 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:51:57.549948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a9e407bc-47a5-4c05-91a1-1db40e6c0e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:51:57.620875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 633/1275] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:51:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:51:57.888669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4c63-9a16-4717-8615-6f545e857f5c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:57.993018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4c63-9a16-4717-8615-6f545e857f5c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:58.010628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:58.029489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:58.030116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:58.030391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:58.030662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:58.041420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 17:51:58.046225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:58.047299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 17:51:58.048490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:58.052301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 643/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Tue Sep 13 17:51:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:58.053638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e407bc-47a5-4c05-91a1-1db40e6c0e76 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:58.143853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:58.164981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e407bc-47a5-4c05-91a1-1db40e6c0e76 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:58.203816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:58.204521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:58.204738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:58.205016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:58.205701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:58.206001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:58.207396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:58.216659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 634/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Tue Sep 13 17:51:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:51:58.222996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:58.331548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580e0d-061b-4b97-a6cb-ba4309287542 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:58.369077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:58.369712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:58.369991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:58.370264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:58.380803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:58.382264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:58.397410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:58.398512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:58.399409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:58.399409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:58.399409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:58.399409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 644/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:58.404612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:58.514594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e84107-f284-4073-835b-9fb164b63fc5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:58.549826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:58.550537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:58.550698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:58.550966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:58.551616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:58.551918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:58.553392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:58.565497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 635/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:58.571498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:58.603424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:58.677529 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e5470-e3ce-4679-90af-82c47a558884 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:58.715380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:58.716041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:58.716320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:58.716625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:58.725832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:58.727310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 17:51:58.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:58.742532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:51:58.743519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 645/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:58.748691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:58.753000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:58.858301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62386a36-304b-48ab-8936-5870e446c35b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:58.894949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:58.895556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:58.895830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:58.896098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:58.896902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:58.897209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:58.898689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:58.916377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 636/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Tue Sep 13 17:51:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:58.922524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:59.019637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:59.030497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab977d-80c8-4c4b-a99f-2ee315609fb9 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:59.064858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:59.065463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:59.065744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:59.066015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:59.137413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:59.138872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:59.234630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:59.245817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 646/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Tue Sep 13 17:51:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 17:51:59.251834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:59.316881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:59.357046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b1e6e-057d-486e-8a39-1eaebe0c5ef8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:59.396280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:59.397070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:59.397344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:59.397613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:59.407777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:59.409281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 17:51:59.424610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:59.430597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 17:51:59.431729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 17:51:59.431729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 17:51:59.431729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:59.431729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 637/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:59.431982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:59.537374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bd45-5edf-462c-adb1-79b34a4439e0 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:59.573846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:59.574518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:59.574623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:59.574890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:59.575544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:59.575851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:59.577254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:59.589118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 647/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Tue Sep 13 17:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:51:59.595049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:59.701380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b142a-6cdd-4835-971e-71890a95ca75 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:59.722670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:59.744528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:59.745168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:59.745445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:59.745722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:59.752560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:59.754017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 17:51:59.765948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:51:59.767041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Sep 13 17:51:59.767953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 638/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Tue Sep 13 17:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:51:59.773582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:51:59.825149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:51:59.881555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579feab7-91f7-4e00-9b35-e4e6ffed84d1 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:51:59.918716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3d3fb-5376-4d36-84e4-d7272858d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:51:59.918981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3d3fb-5376-4d36-84e4-d7272858d9ca None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:51:59.919154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3d3fb-5376-4d36-84e4-d7272858d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:51:59.919428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3d3fb-5376-4d36-84e4-d7272858d9ca None admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:51:59.920265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3d3fb-5376-4d36-84e4-d7272858d9ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:51:59.920623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3d3fb-5376-4d36-84e4-d7272858d9ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:51:59.922073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3d3fb-5376-4d36-84e4-d7272858d9ca None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:51:59.934471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 648/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Tue Sep 13 17:51:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:51:59.940905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:00.052066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175e2bb-fcc3-41a2-8d03-bd20c561d8a8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:00.090952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438a4b3-0a40-4c38-9334-5decd9939b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:00.091459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438a4b3-0a40-4c38-9334-5decd9939b6b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:00.091737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438a4b3-0a40-4c38-9334-5decd9939b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:00.092010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438a4b3-0a40-4c38-9334-5decd9939b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:00.092774 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438a4b3-0a40-4c38-9334-5decd9939b6b None admin] RBAC: Adding query filter params (role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a, scope.project.id=2b736c723c894226850e560a2dd28985, user.id=c6ce465c1ee84703b7b44d33a91d7017) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:52:00.093048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438a4b3-0a40-4c38-9334-5decd9939b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:00.094534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b438a4b3-0a40-4c38-9334-5decd9939b6b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:00.101135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 639/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Tue Sep 13 17:51:59 2022] GET /identity/v3/role_assignments?user.id=c6ce465c1ee84703b7b44d33a91d7017&scope.project.id=2b736c723c894226850e560a2dd28985&role.id=9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:52:02.249225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a41f3-e8b2-4d09-9746-a3126c5c0bb2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:02.249823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a41f3-e8b2-4d09-9746-a3126c5c0bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:02.249823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a41f3-e8b2-4d09-9746-a3126c5c0bb2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:02.251119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 649/1288] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:52:02.256613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9565fd-1b01-499c-93a6-d833bf10e63b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:02.257065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9565fd-1b01-499c-93a6-d833bf10e63b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:02.257167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9565fd-1b01-499c-93a6-d833bf10e63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:02.276803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fd9565fd-1b01-499c-93a6-d833bf10e63b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:52:02.278017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fd9565fd-1b01-499c-93a6-d833bf10e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:52:02.312143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:02.352872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 640/1289] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:52:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:52:02.617504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a41f3-e8b2-4d09-9746-a3126c5c0bb2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:02.725196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a41f3-e8b2-4d09-9746-a3126c5c0bb2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:02.736238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:02.761589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:02.762583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:02.762859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:02.763122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:02.763691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:02.765104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:52:02.773770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:52:02.774989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:02.776075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:02.779783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 650/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:52:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:52:02.780969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9565fd-1b01-499c-93a6-d833bf10e63b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:02.887825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9565fd-1b01-499c-93a6-d833bf10e63b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:02.922373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6940f-5c86-405e-83ea-9199e1456102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:02.922966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6940f-5c86-405e-83ea-9199e1456102 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:02.923239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6940f-5c86-405e-83ea-9199e1456102 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:02.923502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6940f-5c86-405e-83ea-9199e1456102 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:02.924125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6940f-5c86-405e-83ea-9199e1456102 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:52:02.924418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6940f-5c86-405e-83ea-9199e1456102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:02.925829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6940f-5c86-405e-83ea-9199e1456102 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:02.930765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 641/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 13 17:52:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:52:02.936908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:03.041512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f640-959c-43ab-b96b-f86ef43b84f3 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:03.075668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532ea16-df19-4faf-a8c8-103765ab351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:03.076268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ea16-df19-4faf-a8c8-103765ab351b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:03.076569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ea16-df19-4faf-a8c8-103765ab351b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:03.076840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532ea16-df19-4faf-a8c8-103765ab351b None admin] PATH_INFO: `/v3/services` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:03.077470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ea16-df19-4faf-a8c8-103765ab351b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:52:03.077768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ea16-df19-4faf-a8c8-103765ab351b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:03.079143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532ea16-df19-4faf-a8c8-103765ab351b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:03.083733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 651/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 13 17:52:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:52:05.249429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf963c4-4cc8-454d-a415-0d195b922736 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:05.250017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf963c4-4cc8-454d-a415-0d195b922736 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:05.250017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf963c4-4cc8-454d-a415-0d195b922736 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:05.251342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 642/1293] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:52:05.257644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb400be-0c0f-4c1d-845e-fe73999960fc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:05.258205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb400be-0c0f-4c1d-845e-fe73999960fc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:05.258205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb400be-0c0f-4c1d-845e-fe73999960fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:05.278448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2bb400be-0c0f-4c1d-845e-fe73999960fc None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:52:05.279673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2bb400be-0c0f-4c1d-845e-fe73999960fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:52:05.311779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:05.351488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 652/1294] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:52:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:52:05.616624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf963c4-4cc8-454d-a415-0d195b922736 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:05.649888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:05.724365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf963c4-4cc8-454d-a415-0d195b922736 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:05.759704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee729a9-e0eb-43ff-bf68-3465e522d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:05.760313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee729a9-e0eb-43ff-bf68-3465e522d8c8 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:05.760629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee729a9-e0eb-43ff-bf68-3465e522d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:05.760895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee729a9-e0eb-43ff-bf68-3465e522d8c8 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:05.761546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee729a9-e0eb-43ff-bf68-3465e522d8c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:05.763290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee729a9-e0eb-43ff-bf68-3465e522d8c8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:05.779485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 643/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Tue Sep 13 17:52:05 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:52:07.956594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a123aab-6611-44da-a05a-3b14e1c448aa None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:07.957194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a123aab-6611-44da-a05a-3b14e1c448aa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:07.957194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a123aab-6611-44da-a05a-3b14e1c448aa None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:07.958483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 653/1296] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:52:07.964201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75199d99-3f2d-432b-ae5b-e3aa4d9a09ce None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:07.964716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75199d99-3f2d-432b-ae5b-e3aa4d9a09ce None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:07.964716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75199d99-3f2d-432b-ae5b-e3aa4d9a09ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:07.984361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-75199d99-3f2d-432b-ae5b-e3aa4d9a09ce None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:52:07.985531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-75199d99-3f2d-432b-ae5b-e3aa4d9a09ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:52:08.062009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 644/1297] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:52:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:52:08.326360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a123aab-6611-44da-a05a-3b14e1c448aa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:08.431076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a123aab-6611-44da-a05a-3b14e1c448aa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:08.465778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:08.466394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:08.466672 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:08.466942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:08.467530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:08.468940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:52:08.477939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:52:08.479135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:08.480172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 654/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:52:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:52:08.486212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75199d99-3f2d-432b-ae5b-e3aa4d9a09ce None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:08.505061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:08.590690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75199d99-3f2d-432b-ae5b-e3aa4d9a09ce None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:08.626471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:08.627083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:08.627357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:08.627624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:08.628242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:52:08.628570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:08.629955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:08.635428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 645/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 13 17:52:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:52:08.642310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:08.677177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:08.750619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dcfff-6e91-404a-b8f7-cd05cbe7dc77 None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:08.785841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724ea09-d9cc-4505-9397-4ba4c50e3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:08.786451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ea09-d9cc-4505-9397-4ba4c50e3a4b None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:08.786728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ea09-d9cc-4505-9397-4ba4c50e3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:08.787028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ea09-d9cc-4505-9397-4ba4c50e3a4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:08.787821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724ea09-d9cc-4505-9397-4ba4c50e3a4b None admin] RBAC: Adding query filter params (interface=public, service_id=abba178647aa4383bb49e958ce032d3a, region_id=RegionOne) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:52:08.788129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724ea09-d9cc-4505-9397-4ba4c50e3a4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:08.789541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724ea09-d9cc-4505-9397-4ba4c50e3a4b None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:08.794757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 655/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Tue Sep 13 17:52:08 2022] GET /identity/v3/endpoints?service_id=abba178647aa4383bb49e958ce032d3a&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:52:08.800979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:08.886123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:08.906268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552aefb6-d9a3-4bcb-9ad4-123c895ed6b0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:08.942299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be5d90-cb39-41c8-8b60-168f98e2f298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:08.942910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5d90-cb39-41c8-8b60-168f98e2f298 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:08.943181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5d90-cb39-41c8-8b60-168f98e2f298 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:08.943507 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5d90-cb39-41c8-8b60-168f98e2f298 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:08.944119 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be5d90-cb39-41c8-8b60-168f98e2f298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:08.945529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be5d90-cb39-41c8-8b60-168f98e2f298 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:08.952460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 646/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Tue Sep 13 17:52:08 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:52:11.064837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87d175-9d4a-41e0-9c59-a7531786cc81 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:11.065428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87d175-9d4a-41e0-9c59-a7531786cc81 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:11.065428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87d175-9d4a-41e0-9c59-a7531786cc81 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:11.066722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 656/1302] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:52:11.072110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813598f2-2354-4faa-9172-5c6dddc5e9cd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:11.072571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813598f2-2354-4faa-9172-5c6dddc5e9cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:11.072678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813598f2-2354-4faa-9172-5c6dddc5e9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:11.093009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-813598f2-2354-4faa-9172-5c6dddc5e9cd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:52:11.094292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-813598f2-2354-4faa-9172-5c6dddc5e9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:52:11.169253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 647/1303] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:52:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 17:52:11.432563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87d175-9d4a-41e0-9c59-a7531786cc81 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:11.529534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:11.537020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87d175-9d4a-41e0-9c59-a7531786cc81 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:11.571101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:11.571716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:11.571992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:11.572257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] PATH_INFO: `/v3/services/placement` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:11.572864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:11.574233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 17:52:11.583152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 17:52:11.584371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application with Lock( Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self._enter() Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application return self.creator() Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application created_value = creator( Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: ERROR keystone.server.flask.application Sep 13 17:52:11.585260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 657/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Tue Sep 13 17:52:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 17:52:11.590631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813598f2-2354-4faa-9172-5c6dddc5e9cd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:11.671899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:11.697619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813598f2-2354-4faa-9172-5c6dddc5e9cd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:11.733048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dcdbf-7d72-49f8-b974-f7b8c6471169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:11.733593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dcdbf-7d72-49f8-b974-f7b8c6471169 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:11.733869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dcdbf-7d72-49f8-b974-f7b8c6471169 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:11.734144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dcdbf-7d72-49f8-b974-f7b8c6471169 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:11.734781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dcdbf-7d72-49f8-b974-f7b8c6471169 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:52:11.735083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dcdbf-7d72-49f8-b974-f7b8c6471169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:11.736498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dcdbf-7d72-49f8-b974-f7b8c6471169 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:11.741956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 648/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Tue Sep 13 17:52:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:52:11.747986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:52:11.850514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4834a8-9dd3-451d-a4e0-fb4d80e26dfa None admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:52:11.887586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512027b5-7306-49b6-99ca-dd29c9138709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:52:11.888195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512027b5-7306-49b6-99ca-dd29c9138709 None admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:52:11.888501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512027b5-7306-49b6-99ca-dd29c9138709 None admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:52:11.888771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512027b5-7306-49b6-99ca-dd29c9138709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:52:11.889436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512027b5-7306-49b6-99ca-dd29c9138709 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:52:11.890802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512027b5-7306-49b6-99ca-dd29c9138709 None admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:52:11.912795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 658/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:52:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:52:21.738748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=30 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:52:21.908504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:20.901678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289b04e-8d17-4174-96df-31fdfefb98ca None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:20.902273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289b04e-8d17-4174-96df-31fdfefb98ca None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:20.902273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289b04e-8d17-4174-96df-31fdfefb98ca None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:20.903916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 649/1307] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:53:20.909609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dfb15-4ac1-4879-9bcc-f737fbb0c8d3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:20.910121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dfb15-4ac1-4879-9bcc-f737fbb0c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:20.910206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dfb15-4ac1-4879-9bcc-f737fbb0c8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:21.003628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-476dfb15-4ac1-4879-9bcc-f737fbb0c8d3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:21.004851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-476dfb15-4ac1-4879-9bcc-f737fbb0c8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:21.136455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 659/1308] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:21.436891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee487ff-1752-4d19-ade1-db6a0313cfc0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:21.437377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee487ff-1752-4d19-ade1-db6a0313cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:21.437842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee487ff-1752-4d19-ade1-db6a0313cfc0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:21.440480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 650/1309] 10.222.0.43 () {54 vars in 965 bytes} [Tue Sep 13 17:53:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:21.449541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0bc13-c765-442d-bf5b-5455e5eebf2d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:21.449858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0bc13-c765-442d-bf5b-5455e5eebf2d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:21.450109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0bc13-c765-442d-bf5b-5455e5eebf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:21.564153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-03d0bc13-c765-442d-bf5b-5455e5eebf2d None None] MFA Rules not processed for user `a4b10367002d4f64a82b3e9eca00ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:21.566327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-03d0bc13-c765-442d-bf5b-5455e5eebf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:21.607173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:21.699375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 660/1310] 10.222.0.43 () {58 vars in 1062 bytes} [Tue Sep 13 17:53:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:21.709643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee487ff-1752-4d19-ade1-db6a0313cfc0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:21.711863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8ee487ff-1752-4d19-ade1-db6a0313cfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:21.870161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee487ff-1752-4d19-ade1-db6a0313cfc0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:21.979730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5f99a-6827-4d58-8a4d-a5495646448e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:21.980713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f99a-6827-4d58-8a4d-a5495646448e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:21.981191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f99a-6827-4d58-8a4d-a5495646448e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:21.981664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5f99a-6827-4d58-8a4d-a5495646448e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:21.984725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cdf5f99a-6827-4d58-8a4d-a5495646448e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:22.093205 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:22.192712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5f99a-6827-4d58-8a4d-a5495646448e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:22.197371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5f99a-6827-4d58-8a4d-a5495646448e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:22.230690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 651/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:24.944455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388f35-2d01-46d5-a74a-0991b9d83ca1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:24.945359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388f35-2d01-46d5-a74a-0991b9d83ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:24.945359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388f35-2d01-46d5-a74a-0991b9d83ca1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:24.946986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 661/1312] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:24.953782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8330-c24c-492f-a8fe-98f20e477aa1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:24.954625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8330-c24c-492f-a8fe-98f20e477aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:24.954625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8330-c24c-492f-a8fe-98f20e477aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:25.043057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-30ae8330-c24c-492f-a8fe-98f20e477aa1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:25.044591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30ae8330-c24c-492f-a8fe-98f20e477aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:25.158252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 652/1313] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:25.473583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388f35-2d01-46d5-a74a-0991b9d83ca1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:25.476043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8b388f35-2d01-46d5-a74a-0991b9d83ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:25.591919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388f35-2d01-46d5-a74a-0991b9d83ca1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:25.703989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:25.707445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497337f3-3b22-4257-9d24-4d275df0b854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:25.708323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497337f3-3b22-4257-9d24-4d275df0b854 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:25.708808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497337f3-3b22-4257-9d24-4d275df0b854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:25.709198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497337f3-3b22-4257-9d24-4d275df0b854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:25.712321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-497337f3-3b22-4257-9d24-4d275df0b854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:25.874310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497337f3-3b22-4257-9d24-4d275df0b854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:25.875809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497337f3-3b22-4257-9d24-4d275df0b854 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:25.898342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 662/1314] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 17:53:28.483053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4203-f1b2-4110-a3d7-7a5cfc85ba88 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:28.483595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4203-f1b2-4110-a3d7-7a5cfc85ba88 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:28.483595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4203-f1b2-4110-a3d7-7a5cfc85ba88 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:28.484898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 653/1315] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:28.490867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d7b8-a2ab-40a8-a285-f5549e76ecc1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:28.491385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d7b8-a2ab-40a8-a285-f5549e76ecc1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:28.491385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70d7b8-a2ab-40a8-a285-f5549e76ecc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:28.575946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bc70d7b8-a2ab-40a8-a285-f5549e76ecc1 None None] MFA Rules not processed for user `2fd61952074f49b9bed58536e54de503`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:28.577221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bc70d7b8-a2ab-40a8-a285-f5549e76ecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:28.641156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:28.688173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 663/1316] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:28.969603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4203-f1b2-4110-a3d7-7a5cfc85ba88 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:28.971167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0fca4203-f1b2-4110-a3d7-7a5cfc85ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:28.982092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:29.078235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4203-f1b2-4110-a3d7-7a5cfc85ba88 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:29.158528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff030b-a936-4b8e-86f3-b9e05ec166a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:29.159172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff030b-a936-4b8e-86f3-b9e05ec166a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:29.159458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff030b-a936-4b8e-86f3-b9e05ec166a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:29.159697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff030b-a936-4b8e-86f3-b9e05ec166a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:29.161328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-67ff030b-a936-4b8e-86f3-b9e05ec166a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:29.225310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:29.276003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff030b-a936-4b8e-86f3-b9e05ec166a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:29.277499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff030b-a936-4b8e-86f3-b9e05ec166a4 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:29.298117 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 654/1317] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:32.089740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939dd42-9f22-45f2-85f3-869b64458c44 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:32.090415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939dd42-9f22-45f2-85f3-869b64458c44 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:32.090415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939dd42-9f22-45f2-85f3-869b64458c44 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:32.091642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 664/1318] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:32.097288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48baf6-6b2e-43f0-9ff2-91be06059ce9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:32.097751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48baf6-6b2e-43f0-9ff2-91be06059ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:32.097854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48baf6-6b2e-43f0-9ff2-91be06059ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:32.173527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7a48baf6-6b2e-43f0-9ff2-91be06059ce9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:32.174767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7a48baf6-6b2e-43f0-9ff2-91be06059ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:32.275213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 655/1319] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:32.573759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c1049-0167-4282-9e07-6453e29a6d9b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:32.574031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c1049-0167-4282-9e07-6453e29a6d9b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:32.574307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c1049-0167-4282-9e07-6453e29a6d9b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:32.575660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 665/1320] 10.222.0.43 () {54 vars in 965 bytes} [Tue Sep 13 17:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:32.581722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594396-5b01-4db4-aaea-2fa320e565bd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:32.581998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594396-5b01-4db4-aaea-2fa320e565bd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:32.582284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1594396-5b01-4db4-aaea-2fa320e565bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:32.605262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:32.662398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d1594396-5b01-4db4-aaea-2fa320e565bd None None] MFA Rules not processed for user `a4b10367002d4f64a82b3e9eca00ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:32.663610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d1594396-5b01-4db4-aaea-2fa320e565bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:32.747714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 656/1321] 10.222.0.43 () {58 vars in 1062 bytes} [Tue Sep 13 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:53:32.757281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c1049-0167-4282-9e07-6453e29a6d9b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:32.758878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-343c1049-0167-4282-9e07-6453e29a6d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:32.869643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c1049-0167-4282-9e07-6453e29a6d9b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:32.945921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7c775-77a8-443a-a638-5a3f9f6a454d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:32.946532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7c775-77a8-443a-a638-5a3f9f6a454d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:32.946808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7c775-77a8-443a-a638-5a3f9f6a454d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:32.947060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7c775-77a8-443a-a638-5a3f9f6a454d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:32.948711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c1a7c775-77a8-443a-a638-5a3f9f6a454d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:32.955883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:33.062278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7c775-77a8-443a-a638-5a3f9f6a454d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:33.063731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7c775-77a8-443a-a638-5a3f9f6a454d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:33.082641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 666/1322] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:36.382336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f93ef-2068-4000-82f5-17ba0ed4099c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:36.382895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f93ef-2068-4000-82f5-17ba0ed4099c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:36.382895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f93ef-2068-4000-82f5-17ba0ed4099c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:36.384459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 657/1323] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:53:36.390551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6b75-04bb-4af7-9c71-6f63558156c3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:36.391145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6b75-04bb-4af7-9c71-6f63558156c3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:36.391145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d6b75-04bb-4af7-9c71-6f63558156c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:36.472041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-524d6b75-04bb-4af7-9c71-6f63558156c3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:36.473239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-524d6b75-04bb-4af7-9c71-6f63558156c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:36.546934 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:36.595055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 667/1324] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:36.873502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f93ef-2068-4000-82f5-17ba0ed4099c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:36.875323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b9f93ef-2068-4000-82f5-17ba0ed4099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:37.002526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f93ef-2068-4000-82f5-17ba0ed4099c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:37.015331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:37.082615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72382438-29bc-4167-b8fc-f5b210b2c043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:37.083292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382438-29bc-4167-b8fc-f5b210b2c043 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:37.083565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382438-29bc-4167-b8fc-f5b210b2c043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:37.083826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382438-29bc-4167-b8fc-f5b210b2c043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:37.085480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72382438-29bc-4167-b8fc-f5b210b2c043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:37.205929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72382438-29bc-4167-b8fc-f5b210b2c043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:37.207379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72382438-29bc-4167-b8fc-f5b210b2c043 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:37.226920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 658/1325] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:39.456546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b01343-4e27-4cca-9f4e-7e127d12da39 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:39.457132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b01343-4e27-4cca-9f4e-7e127d12da39 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:39.457132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b01343-4e27-4cca-9f4e-7e127d12da39 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:39.458609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 668/1326] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:39.464974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d824a-ce6b-42af-9020-a11b1b6d2dfd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:39.465564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d824a-ce6b-42af-9020-a11b1b6d2dfd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:39.465710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d824a-ce6b-42af-9020-a11b1b6d2dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:39.520266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:39.540251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ef2d824a-ce6b-42af-9020-a11b1b6d2dfd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:39.541481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef2d824a-ce6b-42af-9020-a11b1b6d2dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:39.653505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 659/1327] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:39.934298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b01343-4e27-4cca-9f4e-7e127d12da39 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:39.935873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b7b01343-4e27-4cca-9f4e-7e127d12da39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:40.049191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b01343-4e27-4cca-9f4e-7e127d12da39 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:40.131467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae13eaa-200f-44ed-b28d-3f83cec5e69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:40.132091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae13eaa-200f-44ed-b28d-3f83cec5e69a service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:40.132368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae13eaa-200f-44ed-b28d-3f83cec5e69a service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:40.132668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae13eaa-200f-44ed-b28d-3f83cec5e69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:40.134440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9ae13eaa-200f-44ed-b28d-3f83cec5e69a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:40.155595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:40.264301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae13eaa-200f-44ed-b28d-3f83cec5e69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:40.265817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae13eaa-200f-44ed-b28d-3f83cec5e69a service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:40.286754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 669/1328] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:42.548840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131abfd-c03d-40c6-9556-b33bca2c8156 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:42.549352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131abfd-c03d-40c6-9556-b33bca2c8156 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:42.549352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131abfd-c03d-40c6-9556-b33bca2c8156 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:42.550774 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 660/1329] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:42.557074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf73fc-754a-4643-9bec-10c5c9997efb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:42.557737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf73fc-754a-4643-9bec-10c5c9997efb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:42.557737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf73fc-754a-4643-9bec-10c5c9997efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:42.633612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-65bf73fc-754a-4643-9bec-10c5c9997efb None None] MFA Rules not processed for user `2fd61952074f49b9bed58536e54de503`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:42.634863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-65bf73fc-754a-4643-9bec-10c5c9997efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:42.706448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:42.744360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 670/1330] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:43.025698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131abfd-c03d-40c6-9556-b33bca2c8156 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:43.027248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6131abfd-c03d-40c6-9556-b33bca2c8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:43.146904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131abfd-c03d-40c6-9556-b33bca2c8156 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:43.171170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:43.242006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feed2ee-040a-4898-9c8f-43482cd52738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:43.242631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feed2ee-040a-4898-9c8f-43482cd52738 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:43.242905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feed2ee-040a-4898-9c8f-43482cd52738 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:43.243168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feed2ee-040a-4898-9c8f-43482cd52738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:43.244875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1feed2ee-040a-4898-9c8f-43482cd52738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:43.372518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feed2ee-040a-4898-9c8f-43482cd52738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:43.373905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feed2ee-040a-4898-9c8f-43482cd52738 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:43.393109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 661/1331] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:45.839774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c7462-f57a-43c1-aa4c-4c4035dbc63f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:45.840340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c7462-f57a-43c1-aa4c-4c4035dbc63f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:45.840340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c7462-f57a-43c1-aa4c-4c4035dbc63f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:45.842255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 671/1332] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:45.847496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7739-28c7-4040-b9b5-19b9832b4a40 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:45.847929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7739-28c7-4040-b9b5-19b9832b4a40 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:45.848535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7739-28c7-4040-b9b5-19b9832b4a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:45.918479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:45.922459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c03a7739-28c7-4040-b9b5-19b9832b4a40 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:45.924716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c03a7739-28c7-4040-b9b5-19b9832b4a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:46.029651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 662/1333] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:46.317086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c7462-f57a-43c1-aa4c-4c4035dbc63f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:46.318763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a53c7462-f57a-43c1-aa4c-4c4035dbc63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:46.432234 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c7462-f57a-43c1-aa4c-4c4035dbc63f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:46.508945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a753c-1da5-47fc-99a0-68b4556bc1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:46.509570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a753c-1da5-47fc-99a0-68b4556bc1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:46.509827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a753c-1da5-47fc-99a0-68b4556bc1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:46.510094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a753c-1da5-47fc-99a0-68b4556bc1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:46.512027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-494a753c-1da5-47fc-99a0-68b4556bc1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:46.539366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:46.631569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a753c-1da5-47fc-99a0-68b4556bc1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:46.632967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a753c-1da5-47fc-99a0-68b4556bc1b0 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:46.653108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 672/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:48.881021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a02c-6fed-4d25-945b-f99d18271da7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:48.881585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a02c-6fed-4d25-945b-f99d18271da7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:48.881585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a02c-6fed-4d25-945b-f99d18271da7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:48.882948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 663/1335] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:48.888953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f928a-ecf0-4e62-b06c-d19594158549 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:48.889409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f928a-ecf0-4e62-b06c-d19594158549 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:48.889501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f928a-ecf0-4e62-b06c-d19594158549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:48.959686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-018f928a-ecf0-4e62-b06c-d19594158549 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:48.960963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-018f928a-ecf0-4e62-b06c-d19594158549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:49.036091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:49.058748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 673/1336] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:49.346653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354a02c-6fed-4d25-945b-f99d18271da7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:49.348278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e354a02c-6fed-4d25-945b-f99d18271da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:49.458168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354a02c-6fed-4d25-945b-f99d18271da7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:49.488189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:49.536126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00485dea-47f0-42f4-8755-df5ecc606744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:49.536768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485dea-47f0-42f4-8755-df5ecc606744 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:49.537040 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485dea-47f0-42f4-8755-df5ecc606744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:49.537308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485dea-47f0-42f4-8755-df5ecc606744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:49.538928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-00485dea-47f0-42f4-8755-df5ecc606744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:49.656700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00485dea-47f0-42f4-8755-df5ecc606744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:49.658070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00485dea-47f0-42f4-8755-df5ecc606744 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:49.677000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 664/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:52.222616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352edf-ef8d-4c54-b6e0-79ffd8bae893 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:52.223205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352edf-ef8d-4c54-b6e0-79ffd8bae893 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:52.223301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352edf-ef8d-4c54-b6e0-79ffd8bae893 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:52.224682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 674/1338] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:52.230038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f052-39df-4025-ad9c-66564523839f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:52.230594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f052-39df-4025-ad9c-66564523839f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:52.230594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f052-39df-4025-ad9c-66564523839f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:52.302044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f3f4f052-39df-4025-ad9c-66564523839f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:52.303274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f3f4f052-39df-4025-ad9c-66564523839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:52.312570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:52.407179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 665/1339] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:53:52.690440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352edf-ef8d-4c54-b6e0-79ffd8bae893 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:52.692009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-69352edf-ef8d-4c54-b6e0-79ffd8bae893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:52.806107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352edf-ef8d-4c54-b6e0-79ffd8bae893 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:52.882549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad2dda-4aca-4823-9bd9-6b42169ea336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:52.883162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad2dda-4aca-4823-9bd9-6b42169ea336 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:52.883440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad2dda-4aca-4823-9bd9-6b42169ea336 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:52.883709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad2dda-4aca-4823-9bd9-6b42169ea336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:52.885453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-33ad2dda-4aca-4823-9bd9-6b42169ea336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:52.922412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:52.997014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad2dda-4aca-4823-9bd9-6b42169ea336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:52.998365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad2dda-4aca-4823-9bd9-6b42169ea336 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:53.017005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 675/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:53:57.422780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffc095-a265-4f8b-bc51-2c71c9006026 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:57.424089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffc095-a265-4f8b-bc51-2c71c9006026 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:57.424089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffc095-a265-4f8b-bc51-2c71c9006026 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:57.424699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 666/1341] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:53:57.430732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b1384-dba9-4b08-b652-5043791ff8f1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:57.431287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b1384-dba9-4b08-b652-5043791ff8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:57.431287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b1384-dba9-4b08-b652-5043791ff8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:57.509001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-051b1384-dba9-4b08-b652-5043791ff8f1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:53:57.510262 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-051b1384-dba9-4b08-b652-5043791ff8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:57.600961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:57.613595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 676/1342] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:53:57.890508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffc095-a265-4f8b-bc51-2c71c9006026 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:53:57.892140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c5ffc095-a265-4f8b-bc51-2c71c9006026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:58.003700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffc095-a265-4f8b-bc51-2c71c9006026 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:53:58.045342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:53:58.088637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240054c8-d2ec-4759-af4f-4a00b8daedb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:53:58.089267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240054c8-d2ec-4759-af4f-4a00b8daedb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:53:58.089540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240054c8-d2ec-4759-af4f-4a00b8daedb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:53:58.089807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240054c8-d2ec-4759-af4f-4a00b8daedb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:53:58.091428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-240054c8-d2ec-4759-af4f-4a00b8daedb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:53:58.225288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240054c8-d2ec-4759-af4f-4a00b8daedb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:53:58.226907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240054c8-d2ec-4759-af4f-4a00b8daedb8 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:53:58.247121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 667/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:01.113265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f824c-f4c2-41ad-9a19-74f06dc8fb2d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:01.113880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f824c-f4c2-41ad-9a19-74f06dc8fb2d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:01.113880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f824c-f4c2-41ad-9a19-74f06dc8fb2d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:01.115244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 677/1344] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:01.121552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b72d38-6628-4130-8a01-177ba2cb650f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:01.122154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b72d38-6628-4130-8a01-177ba2cb650f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:01.122154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b72d38-6628-4130-8a01-177ba2cb650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:01.206792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-28b72d38-6628-4130-8a01-177ba2cb650f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:01.208644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-28b72d38-6628-4130-8a01-177ba2cb650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:01.229501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:01.313575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 668/1345] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:01.593106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f824c-f4c2-41ad-9a19-74f06dc8fb2d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:01.594726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d36f824c-f4c2-41ad-9a19-74f06dc8fb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:01.715220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f824c-f4c2-41ad-9a19-74f06dc8fb2d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:01.795889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13e64c-b065-4ff1-b9b4-a86d8d28124d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:01.796515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13e64c-b065-4ff1-b9b4-a86d8d28124d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:01.796802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13e64c-b065-4ff1-b9b4-a86d8d28124d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:01.797069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13e64c-b065-4ff1-b9b4-a86d8d28124d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:01.798653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2f13e64c-b065-4ff1-b9b4-a86d8d28124d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:01.851740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:01.922692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13e64c-b065-4ff1-b9b4-a86d8d28124d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:01.924132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13e64c-b065-4ff1-b9b4-a86d8d28124d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:01.943753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 678/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:06.584156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a2bab-069e-44b9-b7c9-9264f7d49653 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:06.584961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a2bab-069e-44b9-b7c9-9264f7d49653 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:06.584961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a2bab-069e-44b9-b7c9-9264f7d49653 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:06.587931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 669/1347] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:06.593570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ac8b-0b1e-49e6-874f-1fca26248f0d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:06.594090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ac8b-0b1e-49e6-874f-1fca26248f0d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:06.594208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133ac8b-0b1e-49e6-874f-1fca26248f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:06.685387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e133ac8b-0b1e-49e6-874f-1fca26248f0d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:06.687003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e133ac8b-0b1e-49e6-874f-1fca26248f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:06.815563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:06.823730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 679/1348] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:54:07.097685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a2bab-069e-44b9-b7c9-9264f7d49653 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:07.099249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-402a2bab-069e-44b9-b7c9-9264f7d49653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:07.211368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a2bab-069e-44b9-b7c9-9264f7d49653 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:07.261237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:07.288392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2fc2-0a05-46c7-b04a-7a56f767780d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:07.289033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2fc2-0a05-46c7-b04a-7a56f767780d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:07.289303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2fc2-0a05-46c7-b04a-7a56f767780d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:07.289560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2fc2-0a05-46c7-b04a-7a56f767780d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:07.291175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3e7e2fc2-0a05-46c7-b04a-7a56f767780d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:07.414353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2fc2-0a05-46c7-b04a-7a56f767780d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:07.415708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2fc2-0a05-46c7-b04a-7a56f767780d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:07.435819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 670/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 17:54:12.494181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d350a-a144-474e-9903-e98450c3e800 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:12.495731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d350a-a144-474e-9903-e98450c3e800 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:12.496247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d350a-a144-474e-9903-e98450c3e800 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:12.498600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 680/1350] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 17:54:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:12.509376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855c40-2ea2-44a3-91c9-f89b7775bcb7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:12.509376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855c40-2ea2-44a3-91c9-f89b7775bcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:12.509376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03855c40-2ea2-44a3-91c9-f89b7775bcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:12.600189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-03855c40-2ea2-44a3-91c9-f89b7775bcb7 None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:12.602016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-03855c40-2ea2-44a3-91c9-f89b7775bcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:12.636177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:12.782286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 671/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:54:18.171824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca436-df6c-453d-94e5-bf26d2f43069 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:18.172407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca436-df6c-453d-94e5-bf26d2f43069 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:18.172954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca436-df6c-453d-94e5-bf26d2f43069 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:18.175480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 681/1352] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 17:54:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:18.181016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e7472-2b21-460f-b18a-f0604b1468cf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:18.181772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e7472-2b21-460f-b18a-f0604b1468cf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:18.181772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e7472-2b21-460f-b18a-f0604b1468cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:18.315736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1f2e7472-2b21-460f-b18a-f0604b1468cf None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:18.317783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1f2e7472-2b21-460f-b18a-f0604b1468cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:18.422477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:18.493882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 672/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 17:54:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:54:20.376214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e773f-501d-4e1c-a196-4814b5fbefd2 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:20.376997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e773f-501d-4e1c-a196-4814b5fbefd2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:20.376997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e773f-501d-4e1c-a196-4814b5fbefd2 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:20.378190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 682/1354] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 17:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:20.385566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf67c3e-a240-420d-a5cf-6d10519cb150 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:20.386150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf67c3e-a240-420d-a5cf-6d10519cb150 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:20.386693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf67c3e-a240-420d-a5cf-6d10519cb150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:20.511456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1bf67c3e-a240-420d-a5cf-6d10519cb150 None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:20.512598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1bf67c3e-a240-420d-a5cf-6d10519cb150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:20.639607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 673/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 17:54:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:24.429351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a073b-6fd5-4c5b-b26c-157ed7bde0eb None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:24.430376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a073b-6fd5-4c5b-b26c-157ed7bde0eb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:24.430935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a073b-6fd5-4c5b-b26c-157ed7bde0eb None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:24.432974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 683/1356] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:24.444167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f570ae-7a67-4c51-bfe3-3d4a4bbf0a18 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:24.444728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f570ae-7a67-4c51-bfe3-3d4a4bbf0a18 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:24.445121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f570ae-7a67-4c51-bfe3-3d4a4bbf0a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:24.514361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:24.565861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-37f570ae-7a67-4c51-bfe3-3d4a4bbf0a18 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:24.567250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-37f570ae-7a67-4c51-bfe3-3d4a4bbf0a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:24.712891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 674/1357] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:25.011312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ae8a7-572f-4d62-8978-ab497f3cefbe None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:25.011900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ae8a7-572f-4d62-8978-ab497f3cefbe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:25.012364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ae8a7-572f-4d62-8978-ab497f3cefbe None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:25.014189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 684/1358] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 13 17:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:25.022910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460559c4-d818-4f20-b76f-de47996a60d0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:25.023705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460559c4-d818-4f20-b76f-de47996a60d0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:25.024899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460559c4-d818-4f20-b76f-de47996a60d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:25.128538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-460559c4-d818-4f20-b76f-de47996a60d0 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:25.129515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-460559c4-d818-4f20-b76f-de47996a60d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:25.196868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:25.283999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 675/1359] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 13 17:54:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:54:25.293644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ae8a7-572f-4d62-8978-ab497f3cefbe None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:25.295189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1f5ae8a7-572f-4d62-8978-ab497f3cefbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:25.494761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ae8a7-572f-4d62-8978-ab497f3cefbe None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:25.588593 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:25.632268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62541380-b4ea-4cf1-9be6-3125c7ea904d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:25.633010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541380-b4ea-4cf1-9be6-3125c7ea904d service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:25.633281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541380-b4ea-4cf1-9be6-3125c7ea904d service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:25.633574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62541380-b4ea-4cf1-9be6-3125c7ea904d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:25.635774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-62541380-b4ea-4cf1-9be6-3125c7ea904d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:25.759219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62541380-b4ea-4cf1-9be6-3125c7ea904d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:25.761554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62541380-b4ea-4cf1-9be6-3125c7ea904d service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:25.782297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 685/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:28.956143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc88cb-1109-4168-b18f-84ae0dfd829e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:28.956807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc88cb-1109-4168-b18f-84ae0dfd829e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:28.956908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc88cb-1109-4168-b18f-84ae0dfd829e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:28.958773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 676/1361] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:28.965332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97698-bad7-40c4-87c4-1293f30d43d1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:28.965964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97698-bad7-40c4-87c4-1293f30d43d1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:28.965964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d97698-bad7-40c4-87c4-1293f30d43d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:29.057652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b0d97698-bad7-40c4-87c4-1293f30d43d1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:29.059239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b0d97698-bad7-40c4-87c4-1293f30d43d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:29.126380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:29.207498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 686/1362] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:29.504930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc88cb-1109-4168-b18f-84ae0dfd829e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:29.506882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f2dc88cb-1109-4168-b18f-84ae0dfd829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:29.706547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc88cb-1109-4168-b18f-84ae0dfd829e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:29.718604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:29.868008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29f496-c181-432b-80d1-aa0e4d83831f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:29.868972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f496-c181-432b-80d1-aa0e4d83831f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:29.869246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f496-c181-432b-80d1-aa0e4d83831f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:29.869634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f496-c181-432b-80d1-aa0e4d83831f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:29.871325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5d29f496-c181-432b-80d1-aa0e4d83831f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:30.014509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29f496-c181-432b-80d1-aa0e4d83831f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:30.016151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29f496-c181-432b-80d1-aa0e4d83831f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:30.039825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 677/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:32.747915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7215ae2-cdd9-4f2b-b787-ac26fe8e6ef0 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:32.748650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7215ae2-cdd9-4f2b-b787-ac26fe8e6ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:32.748798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7215ae2-cdd9-4f2b-b787-ac26fe8e6ef0 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:32.752601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 687/1364] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:32.756492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060ed59-c47b-4071-9105-c070181e1b91 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:32.757254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060ed59-c47b-4071-9105-c070181e1b91 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:32.757254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060ed59-c47b-4071-9105-c070181e1b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:32.775547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:32.835669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d060ed59-c47b-4071-9105-c070181e1b91 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:32.836942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d060ed59-c47b-4071-9105-c070181e1b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:32.954977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 678/1365] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:54:33.241645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04221422-7d5f-458c-9300-915c6fd41bcf None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:33.241954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04221422-7d5f-458c-9300-915c6fd41bcf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:33.242194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04221422-7d5f-458c-9300-915c6fd41bcf None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:33.243732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 688/1366] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 13 17:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:33.249671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2463aa-3c27-499e-9eba-93e8d99cca97 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:33.250004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2463aa-3c27-499e-9eba-93e8d99cca97 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:33.250227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2463aa-3c27-499e-9eba-93e8d99cca97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:33.345738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8e2463aa-3c27-499e-9eba-93e8d99cca97 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:33.347038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8e2463aa-3c27-499e-9eba-93e8d99cca97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:33.392700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:33.490991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 679/1367] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 13 17:54:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:33.499005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04221422-7d5f-458c-9300-915c6fd41bcf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:33.500601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-04221422-7d5f-458c-9300-915c6fd41bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:33.676610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04221422-7d5f-458c-9300-915c6fd41bcf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:33.701832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:33.802373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1da2bf-ada1-460e-a61f-834d52bcf9f2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:33.802966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1da2bf-ada1-460e-a61f-834d52bcf9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:33.802966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1da2bf-ada1-460e-a61f-834d52bcf9f2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:33.804519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 680/1368] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:54:33.810300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92babe6a-9aee-4975-94b1-7fa7f85bc76f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:33.810566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92babe6a-9aee-4975-94b1-7fa7f85bc76f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:33.811231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92babe6a-9aee-4975-94b1-7fa7f85bc76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:33.812702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd6153-29c3-491a-a840-56682c07b9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:33.813332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6153-29c3-491a-a840-56682c07b9fa service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:33.813792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6153-29c3-491a-a840-56682c07b9fa service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:33.814063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd6153-29c3-491a-a840-56682c07b9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:33.815853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-debd6153-29c3-491a-a840-56682c07b9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:33.893978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-92babe6a-9aee-4975-94b1-7fa7f85bc76f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:33.895246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-92babe6a-9aee-4975-94b1-7fa7f85bc76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:33.944079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd6153-29c3-491a-a840-56682c07b9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:33.946463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd6153-29c3-491a-a840-56682c07b9fa service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:33.970941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 689/1369] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 17:54:34.036560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:34.046518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 681/1370] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:34.344791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17a088-62e6-434c-bb1f-140774b7fe93 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:34.345083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17a088-62e6-434c-bb1f-140774b7fe93 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:34.345363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17a088-62e6-434c-bb1f-140774b7fe93 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:34.347100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 690/1371] 10.222.0.43 () {54 vars in 964 bytes} [Tue Sep 13 17:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:34.352446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed71996-f41b-4592-8c0c-cb9302fb8b77 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:34.352792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed71996-f41b-4592-8c0c-cb9302fb8b77 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:34.353045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed71996-f41b-4592-8c0c-cb9302fb8b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:34.500962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3ed71996-f41b-4592-8c0c-cb9302fb8b77 None None] MFA Rules not processed for user `6c9c5205f9344e8c8fcc85c80d6f504b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:34.503873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3ed71996-f41b-4592-8c0c-cb9302fb8b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:34.630938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 682/1372] 10.222.0.43 () {58 vars in 1061 bytes} [Tue Sep 13 17:54:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:34.657176 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17a088-62e6-434c-bb1f-140774b7fe93 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:34.659984 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cf17a088-62e6-434c-bb1f-140774b7fe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:34.710426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:34.807942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17a088-62e6-434c-bb1f-140774b7fe93 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:34.914929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43de996-dce7-4e93-899b-fe356d8d7959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:34.915911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43de996-dce7-4e93-899b-fe356d8d7959 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:34.915911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43de996-dce7-4e93-899b-fe356d8d7959 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:34.916169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43de996-dce7-4e93-899b-fe356d8d7959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:34.917702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d43de996-dce7-4e93-899b-fe356d8d7959 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:35.031587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:35.081780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43de996-dce7-4e93-899b-fe356d8d7959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:35.083807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43de996-dce7-4e93-899b-fe356d8d7959 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:35.108756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 691/1373] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:36.433888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a7758-5fad-4c10-ad95-c48066c5627b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:36.434340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a7758-5fad-4c10-ad95-c48066c5627b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:36.434499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a7758-5fad-4c10-ad95-c48066c5627b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:36.435981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 683/1374] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:36.441753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192bc1a-528f-4eef-897d-1306aa8c94b4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:36.442306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192bc1a-528f-4eef-897d-1306aa8c94b4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:36.442306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192bc1a-528f-4eef-897d-1306aa8c94b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:36.519495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1192bc1a-528f-4eef-897d-1306aa8c94b4 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:36.520747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1192bc1a-528f-4eef-897d-1306aa8c94b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:36.629731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 692/1375] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:36.920713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a7758-5fad-4c10-ad95-c48066c5627b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:36.922332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a08a7758-5fad-4c10-ad95-c48066c5627b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:37.015146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:37.086860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a7758-5fad-4c10-ad95-c48066c5627b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:37.215640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492f117-851f-4711-8553-55f8a92698d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:37.216748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f117-851f-4711-8553-55f8a92698d0 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:37.217185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f117-851f-4711-8553-55f8a92698d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:37.217455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492f117-851f-4711-8553-55f8a92698d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:37.219488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7492f117-851f-4711-8553-55f8a92698d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:37.276037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:37.321102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492f117-851f-4711-8553-55f8a92698d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:37.322261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492f117-851f-4711-8553-55f8a92698d0 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:37.342959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 684/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:39.629618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef71a5-f854-4fee-ab93-ffc409ec058e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:39.630267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef71a5-f854-4fee-ab93-ffc409ec058e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:39.630267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef71a5-f854-4fee-ab93-ffc409ec058e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:39.631684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 693/1377] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:39.637413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49925275-5372-4cda-a6f1-76d1ec05b225 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:39.637930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49925275-5372-4cda-a6f1-76d1ec05b225 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:39.637930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49925275-5372-4cda-a6f1-76d1ec05b225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:39.723527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-49925275-5372-4cda-a6f1-76d1ec05b225 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:39.725193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-49925275-5372-4cda-a6f1-76d1ec05b225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:39.866531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 685/1378] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:40.161497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef71a5-f854-4fee-ab93-ffc409ec058e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:40.164321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9eef71a5-f854-4fee-ab93-ffc409ec058e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:40.233761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:40.416688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef71a5-f854-4fee-ab93-ffc409ec058e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:40.611880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:40.622053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6833c-b251-4311-8881-ed70abac5f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:40.623092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6833c-b251-4311-8881-ed70abac5f6f service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:40.623571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6833c-b251-4311-8881-ed70abac5f6f service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:40.624040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6833c-b251-4311-8881-ed70abac5f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:40.627165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c5c6833c-b251-4311-8881-ed70abac5f6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:40.789425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6833c-b251-4311-8881-ed70abac5f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:40.792615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6833c-b251-4311-8881-ed70abac5f6f service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:40.826223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 694/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:43.596662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3593-527f-4ff9-8886-25ea11edb14c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:43.597603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3593-527f-4ff9-8886-25ea11edb14c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:43.597603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3593-527f-4ff9-8886-25ea11edb14c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:43.599306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 686/1380] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:54:43.606661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada43f-5009-4a91-ad86-40852c7b925d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:43.607503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada43f-5009-4a91-ad86-40852c7b925d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:43.607684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada43f-5009-4a91-ad86-40852c7b925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:43.713377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c5ada43f-5009-4a91-ad86-40852c7b925d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:43.714704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c5ada43f-5009-4a91-ad86-40852c7b925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:43.797197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:43.834052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 695/1381] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:44.139649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3593-527f-4ff9-8886-25ea11edb14c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:44.143617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-568e3593-527f-4ff9-8886-25ea11edb14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:44.189362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:44.379748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3593-527f-4ff9-8886-25ea11edb14c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:44.501655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:44.522823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57edfc-5dd8-4591-ad39-c02b6c17cee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:44.523613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57edfc-5dd8-4591-ad39-c02b6c17cee2 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:44.523968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57edfc-5dd8-4591-ad39-c02b6c17cee2 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:44.524331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57edfc-5dd8-4591-ad39-c02b6c17cee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:44.526041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fe57edfc-5dd8-4591-ad39-c02b6c17cee2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:44.633639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57edfc-5dd8-4591-ad39-c02b6c17cee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:44.636322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57edfc-5dd8-4591-ad39-c02b6c17cee2 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:44.664878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 687/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:47.007774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dffb0-2af1-426a-b264-3a6f2100d5d5 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:47.008395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dffb0-2af1-426a-b264-3a6f2100d5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:47.008395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dffb0-2af1-426a-b264-3a6f2100d5d5 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:47.010314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 696/1383] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:47.019440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4d6b-9c59-4a35-99ea-f9ab471b26c6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:47.020090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4d6b-9c59-4a35-99ea-f9ab471b26c6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:47.020184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4d6b-9c59-4a35-99ea-f9ab471b26c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:47.115321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b63d4d6b-9c59-4a35-99ea-f9ab471b26c6 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:47.117136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b63d4d6b-9c59-4a35-99ea-f9ab471b26c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:47.191746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:47.246711 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 688/1384] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:47.596303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082dffb0-2af1-426a-b264-3a6f2100d5d5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:47.597972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-082dffb0-2af1-426a-b264-3a6f2100d5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:47.787972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082dffb0-2af1-426a-b264-3a6f2100d5d5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:47.869523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:47.947432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991c056-4b15-4cf7-8de2-00590be51863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:47.948123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991c056-4b15-4cf7-8de2-00590be51863 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:47.948447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991c056-4b15-4cf7-8de2-00590be51863 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:47.948728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991c056-4b15-4cf7-8de2-00590be51863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:47.950522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8991c056-4b15-4cf7-8de2-00590be51863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:48.062957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991c056-4b15-4cf7-8de2-00590be51863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:48.064726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991c056-4b15-4cf7-8de2-00590be51863 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:48.085281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 697/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:50.408977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484171e-1f2a-4440-8dfc-b9a80d5c1a9c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:50.409616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484171e-1f2a-4440-8dfc-b9a80d5c1a9c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:50.409724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484171e-1f2a-4440-8dfc-b9a80d5c1a9c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:50.411683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 689/1386] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:54:50.418433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2180e78-8570-45bc-9ec1-f45fddb25b1f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:50.418966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2180e78-8570-45bc-9ec1-f45fddb25b1f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:50.419187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2180e78-8570-45bc-9ec1-f45fddb25b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:50.498726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e2180e78-8570-45bc-9ec1-f45fddb25b1f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:50.500031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e2180e78-8570-45bc-9ec1-f45fddb25b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:50.513454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:50.630747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 698/1387] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:50.961235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484171e-1f2a-4440-8dfc-b9a80d5c1a9c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:50.962891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1484171e-1f2a-4440-8dfc-b9a80d5c1a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:51.125927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484171e-1f2a-4440-8dfc-b9a80d5c1a9c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:51.180535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:51.270105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd293b4-fb58-4ac1-980b-d48d66ca9929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:51.270676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd293b4-fb58-4ac1-980b-d48d66ca9929 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:51.270948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd293b4-fb58-4ac1-980b-d48d66ca9929 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:51.271211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd293b4-fb58-4ac1-980b-d48d66ca9929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:51.272910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-edd293b4-fb58-4ac1-980b-d48d66ca9929 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:51.376694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd293b4-fb58-4ac1-980b-d48d66ca9929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:51.378520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd293b4-fb58-4ac1-980b-d48d66ca9929 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:51.400195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 690/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:53.905310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e8583-00c3-4b15-9109-658222e0225f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:53.906011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e8583-00c3-4b15-9109-658222e0225f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:53.906011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e8583-00c3-4b15-9109-658222e0225f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:53.907567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 699/1389] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:53.915515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:53.916122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:53.916251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:53.995540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:53.999287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:54.001524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:54.003448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdc88c-ff79-4e54-ae99-a394b24b0f26 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:54.003749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdc88c-ff79-4e54-ae99-a394b24b0f26 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:54.004033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdc88c-ff79-4e54-ae99-a394b24b0f26 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:54.005525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 700/1390] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:54:54.011736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe10594-fd79-4f80-9a21-e10fe247789d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:54.012038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe10594-fd79-4f80-9a21-e10fe247789d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:54.012346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe10594-fd79-4f80-9a21-e10fe247789d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:54.107415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5fe10594-fd79-4f80-9a21-e10fe247789d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:54.109243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5fe10594-fd79-4f80-9a21-e10fe247789d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:54.113206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 691/1391] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:54.225077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:54.240481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 701/1392] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:54:54.409194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:54.410619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:54.531598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bfd91-b4e2-4ad4-a410-9762c635e20f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:54.536365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59f7ff-292d-4166-a613-28c0ac486112 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:54.536677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59f7ff-292d-4166-a613-28c0ac486112 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:54.536916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59f7ff-292d-4166-a613-28c0ac486112 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:54.538287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 702/1393] 10.222.0.43 () {54 vars in 966 bytes} [Tue Sep 13 17:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:54:54.544130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e16582-20c1-4d43-adec-22257b7bdf40 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:54.544325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e16582-20c1-4d43-adec-22257b7bdf40 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:54.544633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e16582-20c1-4d43-adec-22257b7bdf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:54.553636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:54.644206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c3e16582-20c1-4d43-adec-22257b7bdf40 None None] MFA Rules not processed for user `2c282d6f237542e3b873100748759794`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:54.645498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c3e16582-20c1-4d43-adec-22257b7bdf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:54.666822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775ca59-c1a7-4d93-b481-f75d95450a9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:54.667730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775ca59-c1a7-4d93-b481-f75d95450a9c service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:54.668139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775ca59-c1a7-4d93-b481-f75d95450a9c service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:54.668722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775ca59-c1a7-4d93-b481-f75d95450a9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:54.671243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4775ca59-c1a7-4d93-b481-f75d95450a9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:54.735845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 703/1394] 10.222.0.43 () {58 vars in 1063 bytes} [Tue Sep 13 17:54:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:54:54.744837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e16582-20c1-4d43-adec-22257b7bdf40 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:54.753429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e16582-20c1-4d43-adec-22257b7bdf40 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:54.771838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775ca59-c1a7-4d93-b481-f75d95450a9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:54.773903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775ca59-c1a7-4d93-b481-f75d95450a9c service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:54.793007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 692/1395] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:54:54.832237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5c175-52cf-4eec-a995-21fda95e2bda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:54.832944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c175-52cf-4eec-a995-21fda95e2bda service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:54.833228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c175-52cf-4eec-a995-21fda95e2bda service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:54.833513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c175-52cf-4eec-a995-21fda95e2bda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:54.835223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-91e5c175-52cf-4eec-a995-21fda95e2bda service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:54.838528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:54.978426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5c175-52cf-4eec-a995-21fda95e2bda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:54.979952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5c175-52cf-4eec-a995-21fda95e2bda service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:55.000621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 704/1396] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 17:54:55.781422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d7d44-11db-40af-b866-54385dbb8a38 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:55.782009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d7d44-11db-40af-b866-54385dbb8a38 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:55.782009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d7d44-11db-40af-b866-54385dbb8a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:55.809853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:55.853280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f06d7d44-11db-40af-b866-54385dbb8a38 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:55.854499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f06d7d44-11db-40af-b866-54385dbb8a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:55.946367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 693/1397] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 17:54:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:55.994012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8383c-c7f8-4a25-b104-fbd6ffcb0dc5 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:55.994592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8383c-c7f8-4a25-b104-fbd6ffcb0dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:55.994592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8383c-c7f8-4a25-b104-fbd6ffcb0dc5 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:55.995950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 705/1398] 10.222.0.43 () {54 vars in 943 bytes} [Tue Sep 13 17:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 17:54:56.003371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6732113-8079-48a5-8cb5-3b2bbc641f63 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:56.003695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6732113-8079-48a5-8cb5-3b2bbc641f63 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:56.003967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6732113-8079-48a5-8cb5-3b2bbc641f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:56.095468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e6732113-8079-48a5-8cb5-3b2bbc641f63 None None] MFA Rules not processed for user `1ec763f1afac469196a5e897587813b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:56.096888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6732113-8079-48a5-8cb5-3b2bbc641f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:56.184036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:56.272555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 694/1399] 10.222.0.43 () {58 vars in 1040 bytes} [Tue Sep 13 17:54:56 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:56.283311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8383c-c7f8-4a25-b104-fbd6ffcb0dc5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:56.285358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7de8383c-c7f8-4a25-b104-fbd6ffcb0dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:56.438732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:56.474753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8383c-c7f8-4a25-b104-fbd6ffcb0dc5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:56.597609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fda57b-59f4-42cc-8063-088566520ae0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:56.598366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fda57b-59f4-42cc-8063-088566520ae0 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:56.598810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fda57b-59f4-42cc-8063-088566520ae0 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:56.599185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fda57b-59f4-42cc-8063-088566520ae0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:56.601219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e8fda57b-59f4-42cc-8063-088566520ae0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:56.708441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fda57b-59f4-42cc-8063-088566520ae0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:56.710215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fda57b-59f4-42cc-8063-088566520ae0 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:56.720093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 706/1400] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 17:54:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 17:54:57.040935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289fc94-d71c-45de-a3a7-be03f8ebd09e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:57.041522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289fc94-d71c-45de-a3a7-be03f8ebd09e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:57.042194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289fc94-d71c-45de-a3a7-be03f8ebd09e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:57.044876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 695/1401] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:54:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:54:57.054606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700fa06-af37-4777-8d4c-2e9fe52c73ba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:57.055176 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700fa06-af37-4777-8d4c-2e9fe52c73ba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:57.055577 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700fa06-af37-4777-8d4c-2e9fe52c73ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:57.069582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:57.150325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f700fa06-af37-4777-8d4c-2e9fe52c73ba None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:54:57.151569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f700fa06-af37-4777-8d4c-2e9fe52c73ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:57.275823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 707/1402] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:54:57.568047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289fc94-d71c-45de-a3a7-be03f8ebd09e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:54:57.569668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0289fc94-d71c-45de-a3a7-be03f8ebd09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:57.775503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289fc94-d71c-45de-a3a7-be03f8ebd09e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:54:57.807884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:54:57.924796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0e9f5-f9ff-4e98-a735-600745e64c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:54:57.925453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0e9f5-f9ff-4e98-a735-600745e64c79 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:54:57.925872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0e9f5-f9ff-4e98-a735-600745e64c79 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:54:57.926362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0e9f5-f9ff-4e98-a735-600745e64c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:54:57.929693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-74f0e9f5-f9ff-4e98-a735-600745e64c79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:54:58.034395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0e9f5-f9ff-4e98-a735-600745e64c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:54:58.036213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0e9f5-f9ff-4e98-a735-600745e64c79 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:54:58.057322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 696/1403] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:00.433655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0af6d-2186-4b10-8dc7-54aacb67647a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:00.434415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0af6d-2186-4b10-8dc7-54aacb67647a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:00.434415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0af6d-2186-4b10-8dc7-54aacb67647a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:00.440281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 708/1404] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:55:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:00.442875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc5ea-ebce-422b-a0d3-6efda1b46bfd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:00.443416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc5ea-ebce-422b-a0d3-6efda1b46bfd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:00.443884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc5ea-ebce-422b-a0d3-6efda1b46bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:00.523909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:00.565347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-98abc5ea-ebce-422b-a0d3-6efda1b46bfd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:00.566654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-98abc5ea-ebce-422b-a0d3-6efda1b46bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:00.708594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 697/1405] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:01.004783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0af6d-2186-4b10-8dc7-54aacb67647a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:01.006362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2cc0af6d-2186-4b10-8dc7-54aacb67647a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:01.117797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:01.207869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0af6d-2186-4b10-8dc7-54aacb67647a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:01.364361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5dc9b1-48cd-44ea-9d6e-3b113cd74c8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:01.365174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dc9b1-48cd-44ea-9d6e-3b113cd74c8a service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:01.365174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dc9b1-48cd-44ea-9d6e-3b113cd74c8a service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:01.366033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dc9b1-48cd-44ea-9d6e-3b113cd74c8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:01.367230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e5dc9b1-48cd-44ea-9d6e-3b113cd74c8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:01.445229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:01.511760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dc9b1-48cd-44ea-9d6e-3b113cd74c8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:01.518145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dc9b1-48cd-44ea-9d6e-3b113cd74c8a service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:01.539699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 709/1406] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:04.156991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e138c-4b4f-4b97-8f51-e94bceb0dd8f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:04.157608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e138c-4b4f-4b97-8f51-e94bceb0dd8f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:04.158008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e138c-4b4f-4b97-8f51-e94bceb0dd8f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:04.166864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 698/1407] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:55:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:04.173652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58096596-1530-40d0-acc7-fb49243fb553 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:04.174242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58096596-1530-40d0-acc7-fb49243fb553 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:04.174865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58096596-1530-40d0-acc7-fb49243fb553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:04.288391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-58096596-1530-40d0-acc7-fb49243fb553 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:04.289791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-58096596-1530-40d0-acc7-fb49243fb553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:04.416562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 710/1408] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:04.709486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e138c-4b4f-4b97-8f51-e94bceb0dd8f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:04.711002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-798e138c-4b4f-4b97-8f51-e94bceb0dd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:04.862692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:04.912778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e138c-4b4f-4b97-8f51-e94bceb0dd8f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:05.071840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfe5c7-7515-4d4d-a4e3-ae74fabba9b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:05.072708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfe5c7-7515-4d4d-a4e3-ae74fabba9b1 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:05.073114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfe5c7-7515-4d4d-a4e3-ae74fabba9b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:05.073436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfe5c7-7515-4d4d-a4e3-ae74fabba9b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:05.075284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6bcfe5c7-7515-4d4d-a4e3-ae74fabba9b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:05.165012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:05.184979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfe5c7-7515-4d4d-a4e3-ae74fabba9b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:05.186669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfe5c7-7515-4d4d-a4e3-ae74fabba9b1 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:05.219349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 699/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:14.412608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:15.216309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=13 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:20.527309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3ecd-8e35-4bf0-a52d-8e4b0e6249a5 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:20.527902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3ecd-8e35-4bf0-a52d-8e4b0e6249a5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:20.527902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba3ecd-8e35-4bf0-a52d-8e4b0e6249a5 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:20.529136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 711/1410] 10.222.0.43 () {54 vars in 918 bytes} [Tue Sep 13 17:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:20.534649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3dbb-c577-4c1a-99d2-71bae00a621b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:20.535133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3dbb-c577-4c1a-99d2-71bae00a621b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:20.535238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3dbb-c577-4c1a-99d2-71bae00a621b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:20.614954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ac8f3dbb-c577-4c1a-99d2-71bae00a621b None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:20.615974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ac8f3dbb-c577-4c1a-99d2-71bae00a621b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:20.729018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 700/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Sep 13 17:55:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 17:55:20.743906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179a880-7196-4262-85e7-8880f5a31f8b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:20.744193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179a880-7196-4262-85e7-8880f5a31f8b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:20.744467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179a880-7196-4262-85e7-8880f5a31f8b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:20.745813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 712/1412] 10.222.0.43 () {54 vars in 918 bytes} [Tue Sep 13 17:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:20.750812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2c0ac-c5f1-488e-a5de-38b7390a9ab2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:20.751104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2c0ac-c5f1-488e-a5de-38b7390a9ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:20.751425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2c0ac-c5f1-488e-a5de-38b7390a9ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:20.826594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:20.830310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0ff2c0ac-c5f1-488e-a5de-38b7390a9ab2 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:20.832378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0ff2c0ac-c5f1-488e-a5de-38b7390a9ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:20.960247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 701/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Sep 13 17:55:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:23.598399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71533c-14f7-4a24-8096-470f256460c5 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:23.599003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71533c-14f7-4a24-8096-470f256460c5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:23.599003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa71533c-14f7-4a24-8096-470f256460c5 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:23.600346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:23.606499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fb159-3dce-4afc-bb36-d4372713ca76 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:23.607125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fb159-3dce-4afc-bb36-d4372713ca76 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:23.607125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fb159-3dce-4afc-bb36-d4372713ca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:23.683549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7d0fb159-3dce-4afc-bb36-d4372713ca76 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:23.685127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7d0fb159-3dce-4afc-bb36-d4372713ca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:23.748173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:23.796687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:24.086534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71533c-14f7-4a24-8096-470f256460c5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:24.088071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fa71533c-14f7-4a24-8096-470f256460c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:24.260267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa71533c-14f7-4a24-8096-470f256460c5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:24.336581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:24.379341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb55abb-9427-4acc-8ab1-4af5c9c26141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:24.379953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb55abb-9427-4acc-8ab1-4af5c9c26141 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:24.380224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb55abb-9427-4acc-8ab1-4af5c9c26141 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:24.380652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb55abb-9427-4acc-8ab1-4af5c9c26141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:24.382243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fbb55abb-9427-4acc-8ab1-4af5c9c26141 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:24.480376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb55abb-9427-4acc-8ab1-4af5c9c26141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:24.481887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb55abb-9427-4acc-8ab1-4af5c9c26141 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:24.500230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 714/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:25.648066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46178ba-14d2-4ab4-acdc-3ff7af6cfa07 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:25.648742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46178ba-14d2-4ab4-acdc-3ff7af6cfa07 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:25.648742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46178ba-14d2-4ab4-acdc-3ff7af6cfa07 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:25.650000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:25.656877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a634d4c-5cd8-40dc-8b9c-e7ee95a58d8a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:25.657418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a634d4c-5cd8-40dc-8b9c-e7ee95a58d8a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:25.657418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a634d4c-5cd8-40dc-8b9c-e7ee95a58d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:25.745548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7a634d4c-5cd8-40dc-8b9c-e7ee95a58d8a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:25.746780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7a634d4c-5cd8-40dc-8b9c-e7ee95a58d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:25.777003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:25.853265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:26.005899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46178ba-14d2-4ab4-acdc-3ff7af6cfa07 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:26.007372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f46178ba-14d2-4ab4-acdc-3ff7af6cfa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:26.174118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46178ba-14d2-4ab4-acdc-3ff7af6cfa07 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:26.225654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:26.306588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f521-379f-4347-93e4-61daea9899bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:26.307206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f521-379f-4347-93e4-61daea9899bd service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:26.307472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f521-379f-4347-93e4-61daea9899bd service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:26.307737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f521-379f-4347-93e4-61daea9899bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:26.309388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c2d0f521-379f-4347-93e4-61daea9899bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:26.404964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f521-379f-4347-93e4-61daea9899bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:26.406606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f521-379f-4347-93e4-61daea9899bd service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:26.425714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 704/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:27.588944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76eff2-da15-4d3f-9f7e-520844a55251 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:27.589534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76eff2-da15-4d3f-9f7e-520844a55251 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:27.589534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76eff2-da15-4d3f-9f7e-520844a55251 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:27.590862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:27.597911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36309e-6321-4fa9-af56-b5744f8cb5be None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:27.598488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36309e-6321-4fa9-af56-b5744f8cb5be None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:27.598488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36309e-6321-4fa9-af56-b5744f8cb5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:27.657556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:27.681931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6e36309e-6321-4fa9-af56-b5744f8cb5be None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:27.683201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6e36309e-6321-4fa9-af56-b5744f8cb5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:27.788307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:27.943758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76eff2-da15-4d3f-9f7e-520844a55251 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:27.946019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dc76eff2-da15-4d3f-9f7e-520844a55251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:28.106686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76eff2-da15-4d3f-9f7e-520844a55251 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:28.119349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:28.225163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c1a85-9b41-43f9-a1c1-bcb6ec75648e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:28.225686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c1a85-9b41-43f9-a1c1-bcb6ec75648e service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:28.225962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c1a85-9b41-43f9-a1c1-bcb6ec75648e service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:28.226232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c1a85-9b41-43f9-a1c1-bcb6ec75648e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:28.227859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bb3c1a85-9b41-43f9-a1c1-bcb6ec75648e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:28.326519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c1a85-9b41-43f9-a1c1-bcb6ec75648e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:28.328920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c1a85-9b41-43f9-a1c1-bcb6ec75648e service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:28.352507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 717/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:29.505193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3149ae-24d3-45fd-a02e-de6873775d87 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:29.505816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3149ae-24d3-45fd-a02e-de6873775d87 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:29.505816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3149ae-24d3-45fd-a02e-de6873775d87 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:29.507078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:29.514397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629317-a633-4f4b-bd18-588b1250a675 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:29.514898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629317-a633-4f4b-bd18-588b1250a675 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:29.514898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629317-a633-4f4b-bd18-588b1250a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:29.540036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:29.600187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-df629317-a633-4f4b-bd18-588b1250a675 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:29.601440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-df629317-a633-4f4b-bd18-588b1250a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:29.706454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:29.856826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3149ae-24d3-45fd-a02e-de6873775d87 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:29.858496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fc3149ae-24d3-45fd-a02e-de6873775d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:29.983115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:30.027314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3149ae-24d3-45fd-a02e-de6873775d87 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:30.162854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb4817-003c-45a2-b240-f80da9b98772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:30.163807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb4817-003c-45a2-b240-f80da9b98772 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:30.164295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb4817-003c-45a2-b240-f80da9b98772 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:30.164831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb4817-003c-45a2-b240-f80da9b98772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:30.167674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bdbb4817-003c-45a2-b240-f80da9b98772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:30.278279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:30.293702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb4817-003c-45a2-b240-f80da9b98772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:30.295351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb4817-003c-45a2-b240-f80da9b98772 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:30.314971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 707/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:31.443690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274d7d9-011f-42ac-904e-0aeabe1c6713 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:31.444332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274d7d9-011f-42ac-904e-0aeabe1c6713 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:31.444332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274d7d9-011f-42ac-904e-0aeabe1c6713 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:31.445671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:31.452563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640d8e-ab2d-455d-839c-d46ef196cb5e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:31.453102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640d8e-ab2d-455d-839c-d46ef196cb5e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:31.453102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640d8e-ab2d-455d-839c-d46ef196cb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:31.531396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5d640d8e-ab2d-455d-839c-d46ef196cb5e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:31.532679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5d640d8e-ab2d-455d-839c-d46ef196cb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:31.634011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:31.785198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274d7d9-011f-42ac-904e-0aeabe1c6713 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:31.786761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d274d7d9-011f-42ac-904e-0aeabe1c6713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:31.876419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:31.944207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274d7d9-011f-42ac-904e-0aeabe1c6713 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:32.068365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe214a-37b8-44c4-9844-5c999d1c4609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:32.069016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe214a-37b8-44c4-9844-5c999d1c4609 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:32.069267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe214a-37b8-44c4-9844-5c999d1c4609 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:32.069572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe214a-37b8-44c4-9844-5c999d1c4609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:32.071193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e1fe214a-37b8-44c4-9844-5c999d1c4609 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:32.121972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:32.171622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe214a-37b8-44c4-9844-5c999d1c4609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:32.173309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe214a-37b8-44c4-9844-5c999d1c4609 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:32.193190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 720/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:33.268614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cdf03-5fcb-4118-80be-5c7fe3168103 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:33.269229 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cdf03-5fcb-4118-80be-5c7fe3168103 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:33.269229 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cdf03-5fcb-4118-80be-5c7fe3168103 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:33.270516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:33.277367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e216e13-e928-4fed-92e8-0e6033f87467 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:33.277920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e216e13-e928-4fed-92e8-0e6033f87467 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:33.277920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e216e13-e928-4fed-92e8-0e6033f87467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:33.358388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4e216e13-e928-4fed-92e8-0e6033f87467 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:33.359620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4e216e13-e928-4fed-92e8-0e6033f87467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:33.462154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:33.610733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cdf03-5fcb-4118-80be-5c7fe3168103 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:33.612258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f80cdf03-5fcb-4118-80be-5c7fe3168103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:33.668211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:33.777599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cdf03-5fcb-4118-80be-5c7fe3168103 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:33.894116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6d739-d2b8-44c8-ac59-137e450d233f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:33.894886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d739-d2b8-44c8-ac59-137e450d233f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:33.895325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d739-d2b8-44c8-ac59-137e450d233f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:33.895739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6d739-d2b8-44c8-ac59-137e450d233f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:33.898517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e7f6d739-d2b8-44c8-ac59-137e450d233f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:33.906977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:34.011105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6d739-d2b8-44c8-ac59-137e450d233f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:34.012780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6d739-d2b8-44c8-ac59-137e450d233f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:34.031765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 710/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:35.106468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de6659-14df-413f-85ce-881f535b6f52 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:35.107056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de6659-14df-413f-85ce-881f535b6f52 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:35.107056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de6659-14df-413f-85ce-881f535b6f52 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:35.108366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:35.115214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76686803-c7a0-4100-9eba-e27e10d4755e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:35.115731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76686803-c7a0-4100-9eba-e27e10d4755e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:35.115731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76686803-c7a0-4100-9eba-e27e10d4755e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:35.199222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-76686803-c7a0-4100-9eba-e27e10d4755e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:35.200532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-76686803-c7a0-4100-9eba-e27e10d4755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:35.273938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:35.304039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:35.452566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de6659-14df-413f-85ce-881f535b6f52 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:35.454303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c2de6659-14df-413f-85ce-881f535b6f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:35.473136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:35.653242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de6659-14df-413f-85ce-881f535b6f52 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:35.751991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:35.778098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3776c-d8d8-4dea-93a9-4bac41ec22ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:35.778717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3776c-d8d8-4dea-93a9-4bac41ec22ba service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:35.778993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3776c-d8d8-4dea-93a9-4bac41ec22ba service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:35.779260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3776c-d8d8-4dea-93a9-4bac41ec22ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:35.780926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a8c3776c-d8d8-4dea-93a9-4bac41ec22ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:35.873752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3776c-d8d8-4dea-93a9-4bac41ec22ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:35.875353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3776c-d8d8-4dea-93a9-4bac41ec22ba service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:35.895696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 723/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:36.970302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc4ef-1756-4031-b6e0-a8c1341d5405 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:36.970923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc4ef-1756-4031-b6e0-a8c1341d5405 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:36.970923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc4ef-1756-4031-b6e0-a8c1341d5405 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:36.972253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:36.979157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a329e48-f8b0-411d-aeb1-78217a48ad5c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:36.979621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a329e48-f8b0-411d-aeb1-78217a48ad5c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:36.979836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a329e48-f8b0-411d-aeb1-78217a48ad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:37.058911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3a329e48-f8b0-411d-aeb1-78217a48ad5c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:37.060134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3a329e48-f8b0-411d-aeb1-78217a48ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:37.115764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:37.179321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:37.330085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc4ef-1756-4031-b6e0-a8c1341d5405 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:37.331575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-923dc4ef-1756-4031-b6e0-a8c1341d5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:37.498732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc4ef-1756-4031-b6e0-a8c1341d5405 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:37.555858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:37.618699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68c72f-4826-4f3d-be67-b9f41f616415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:37.619216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68c72f-4826-4f3d-be67-b9f41f616415 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:37.619491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68c72f-4826-4f3d-be67-b9f41f616415 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:37.619758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68c72f-4826-4f3d-be67-b9f41f616415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:37.621422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b68c72f-4826-4f3d-be67-b9f41f616415 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:37.717853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68c72f-4826-4f3d-be67-b9f41f616415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:37.719505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68c72f-4826-4f3d-be67-b9f41f616415 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:37.741765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 713/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:38.820317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f88c1-e17d-495b-a228-b0444e07cfb8 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:38.820971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f88c1-e17d-495b-a228-b0444e07cfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:38.820971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f88c1-e17d-495b-a228-b0444e07cfb8 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:38.822213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:38.828807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc774-3ee8-4958-a81d-ac4244c00de9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:38.829272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc774-3ee8-4958-a81d-ac4244c00de9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:38.829367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc774-3ee8-4958-a81d-ac4244c00de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:38.911585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9d8cc774-3ee8-4958-a81d-ac4244c00de9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:38.912881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9d8cc774-3ee8-4958-a81d-ac4244c00de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:38.914979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:39.015034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:39.165181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f88c1-e17d-495b-a228-b0444e07cfb8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:39.166754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-392f88c1-e17d-495b-a228-b0444e07cfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:39.325462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f88c1-e17d-495b-a228-b0444e07cfb8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:39.353640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:39.447394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1cd3f-d75a-455e-93e7-b31c5ea921a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:39.447980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1cd3f-d75a-455e-93e7-b31c5ea921a7 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:39.448271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1cd3f-d75a-455e-93e7-b31c5ea921a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:39.448575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1cd3f-d75a-455e-93e7-b31c5ea921a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:39.450184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cbd1cd3f-d75a-455e-93e7-b31c5ea921a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:39.547225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1cd3f-d75a-455e-93e7-b31c5ea921a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:39.548937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1cd3f-d75a-455e-93e7-b31c5ea921a7 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:39.569632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 726/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:41.662349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf72a53-fa19-4f47-8589-85181e9fdb58 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:41.663067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf72a53-fa19-4f47-8589-85181e9fdb58 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:41.663067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf72a53-fa19-4f47-8589-85181e9fdb58 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:41.664255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:41.671274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad6e20-4e2c-466d-b6a3-cdaf4f437360 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:41.672132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad6e20-4e2c-466d-b6a3-cdaf4f437360 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:41.672132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad6e20-4e2c-466d-b6a3-cdaf4f437360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:41.723229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:41.761310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4cad6e20-4e2c-466d-b6a3-cdaf4f437360 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:41.762993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4cad6e20-4e2c-466d-b6a3-cdaf4f437360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:41.879877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:42.165691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf72a53-fa19-4f47-8589-85181e9fdb58 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:42.167256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-eaf72a53-fa19-4f47-8589-85181e9fdb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:42.282326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf72a53-fa19-4f47-8589-85181e9fdb58 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:42.321637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:42.366825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2362d2-c49a-49cf-af09-2f9d7214474c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:42.367441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2362d2-c49a-49cf-af09-2f9d7214474c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:42.367717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2362d2-c49a-49cf-af09-2f9d7214474c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:42.367986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2362d2-c49a-49cf-af09-2f9d7214474c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:42.369651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1e2362d2-c49a-49cf-af09-2f9d7214474c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:42.488900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2362d2-c49a-49cf-af09-2f9d7214474c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:42.490280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2362d2-c49a-49cf-af09-2f9d7214474c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:42.509474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 716/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 17:55:44.895155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3faf29-77ff-4c24-901c-9b8879de56cc None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:44.895826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3faf29-77ff-4c24-901c-9b8879de56cc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:44.895826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3faf29-77ff-4c24-901c-9b8879de56cc None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:44.897177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:44.903497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f26492-d75a-4bca-9cf5-fb3a95d8fa7c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:44.904126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f26492-d75a-4bca-9cf5-fb3a95d8fa7c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:44.904126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f26492-d75a-4bca-9cf5-fb3a95d8fa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:44.987812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-10f26492-d75a-4bca-9cf5-fb3a95d8fa7c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:44.988985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-10f26492-d75a-4bca-9cf5-fb3a95d8fa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:45.005006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:45.101329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:45.393726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3faf29-77ff-4c24-901c-9b8879de56cc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:45.395481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8e3faf29-77ff-4c24-901c-9b8879de56cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:45.508280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3faf29-77ff-4c24-901c-9b8879de56cc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:45.522531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:45.589499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181631c6-7f6a-479f-a797-d7c17dbc0abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:45.590088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181631c6-7f6a-479f-a797-d7c17dbc0abb service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:45.590424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181631c6-7f6a-479f-a797-d7c17dbc0abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:45.590628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181631c6-7f6a-479f-a797-d7c17dbc0abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:45.592246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-181631c6-7f6a-479f-a797-d7c17dbc0abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:45.725375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181631c6-7f6a-479f-a797-d7c17dbc0abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:45.726755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181631c6-7f6a-479f-a797-d7c17dbc0abb service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:45.747188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 729/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:48.221695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d401d-d47c-4b77-8047-ea4f9f6e9a92 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:48.222374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d401d-d47c-4b77-8047-ea4f9f6e9a92 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:48.222374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d401d-d47c-4b77-8047-ea4f9f6e9a92 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:48.223636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:48.231924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd9f51-a983-49d2-adc6-a93cf9ddfe87 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:48.232549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd9f51-a983-49d2-adc6-a93cf9ddfe87 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:48.232549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd9f51-a983-49d2-adc6-a93cf9ddfe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:48.292070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:48.315423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-58cd9f51-a983-49d2-adc6-a93cf9ddfe87 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:48.316702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-58cd9f51-a983-49d2-adc6-a93cf9ddfe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:48.446677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:48.729230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d401d-d47c-4b77-8047-ea4f9f6e9a92 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:48.730800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-223d401d-d47c-4b77-8047-ea4f9f6e9a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:48.839598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d401d-d47c-4b77-8047-ea4f9f6e9a92 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:48.884135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:48.922405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d4a97-d534-4690-8474-21ec28949400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:48.923014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d4a97-d534-4690-8474-21ec28949400 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:48.923281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d4a97-d534-4690-8474-21ec28949400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:48.923594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d4a97-d534-4690-8474-21ec28949400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:48.925236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b55d4a97-d534-4690-8474-21ec28949400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:49.045374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d4a97-d534-4690-8474-21ec28949400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:49.046842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d4a97-d534-4690-8474-21ec28949400 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:49.066416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 719/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:51.347882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112929ff-9c26-4cf4-a2ab-fc8a641fddcd None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:51.348524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112929ff-9c26-4cf4-a2ab-fc8a641fddcd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:51.348524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112929ff-9c26-4cf4-a2ab-fc8a641fddcd None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:51.349781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:51.355716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100aef-b49a-4cad-aadb-fec11d7ea005 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:51.356293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100aef-b49a-4cad-aadb-fec11d7ea005 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:51.356293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100aef-b49a-4cad-aadb-fec11d7ea005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:51.432369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ef100aef-b49a-4cad-aadb-fec11d7ea005 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:51.433662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef100aef-b49a-4cad-aadb-fec11d7ea005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:51.458383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:51.542175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:51.820620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112929ff-9c26-4cf4-a2ab-fc8a641fddcd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:51.822170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-112929ff-9c26-4cf4-a2ab-fc8a641fddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:51.932955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112929ff-9c26-4cf4-a2ab-fc8a641fddcd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:51.953703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:52.010149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8c011-f029-4ce7-a149-5ce0eb0adf44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:52.010681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8c011-f029-4ce7-a149-5ce0eb0adf44 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:52.010953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8c011-f029-4ce7-a149-5ce0eb0adf44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:52.011220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8c011-f029-4ce7-a149-5ce0eb0adf44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:52.012851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7bf8c011-f029-4ce7-a149-5ce0eb0adf44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:52.128060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8c011-f029-4ce7-a149-5ce0eb0adf44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:52.129636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8c011-f029-4ce7-a149-5ce0eb0adf44 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:52.150329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 732/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 17:55:54.470015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c630dc-d44c-4546-82b2-78add98b8281 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:54.470614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c630dc-d44c-4546-82b2-78add98b8281 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:54.470614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c630dc-d44c-4546-82b2-78add98b8281 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:54.472077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:54.478758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2daaf-b030-40e0-9b29-9997c37d1878 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:54.479332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2daaf-b030-40e0-9b29-9997c37d1878 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:54.479332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2daaf-b030-40e0-9b29-9997c37d1878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:54.555569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:54.558827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d7b2daaf-b030-40e0-9b29-9997c37d1878 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:54.560460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d7b2daaf-b030-40e0-9b29-9997c37d1878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:54.670140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:54.955275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c630dc-d44c-4546-82b2-78add98b8281 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:54.957033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-05c630dc-d44c-4546-82b2-78add98b8281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:55.069538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c630dc-d44c-4546-82b2-78add98b8281 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:55.124239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:55.153527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1eb6a-fe69-4706-bfd6-385aae199f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:55.154158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1eb6a-fe69-4706-bfd6-385aae199f1f service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:55.154439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1eb6a-fe69-4706-bfd6-385aae199f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:55.154708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1eb6a-fe69-4706-bfd6-385aae199f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:55.156446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8ed1eb6a-fe69-4706-bfd6-385aae199f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:55.276195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1eb6a-fe69-4706-bfd6-385aae199f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:55.277786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1eb6a-fe69-4706-bfd6-385aae199f1f service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:55.298195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 722/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:55:57.538524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500f10-6fca-40e4-8e4e-f966c1cf67af None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:57.539146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500f10-6fca-40e4-8e4e-f966c1cf67af None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:57.539146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12500f10-6fca-40e4-8e4e-f966c1cf67af None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:57.540457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:55:57.546748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d644e0-1f1d-4853-b23b-835c06fa4124 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:57.547385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d644e0-1f1d-4853-b23b-835c06fa4124 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:57.547385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d644e0-1f1d-4853-b23b-835c06fa4124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:57.622019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-65d644e0-1f1d-4853-b23b-835c06fa4124 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:55:57.623294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-65d644e0-1f1d-4853-b23b-835c06fa4124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:57.653887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:57.725704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:55:58.008399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12500f10-6fca-40e4-8e4e-f966c1cf67af None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:55:58.009980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-12500f10-6fca-40e4-8e4e-f966c1cf67af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:58.115860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12500f10-6fca-40e4-8e4e-f966c1cf67af None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:55:58.146334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:55:58.192944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d5d0c-fe39-479b-b059-80d9052251f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:55:58.193583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5d0c-fe39-479b-b059-80d9052251f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:55:58.193834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5d0c-fe39-479b-b059-80d9052251f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:55:58.194118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d5d0c-fe39-479b-b059-80d9052251f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:55:58.195744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-642d5d0c-fe39-479b-b059-80d9052251f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:55:58.314541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d5d0c-fe39-479b-b059-80d9052251f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:55:58.316067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d5d0c-fe39-479b-b059-80d9052251f1 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:55:58.335374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 735/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:00.542911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86426052-da36-4f6d-b21c-fcdb68153607 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:00.543495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86426052-da36-4f6d-b21c-fcdb68153607 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:00.543495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86426052-da36-4f6d-b21c-fcdb68153607 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:00.544916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:00.551182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5476f5-75c0-4efb-ac96-c7e45d971f3d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:00.551763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5476f5-75c0-4efb-ac96-c7e45d971f3d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:00.551763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5476f5-75c0-4efb-ac96-c7e45d971f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:00.646634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0e5476f5-75c0-4efb-ac96-c7e45d971f3d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:00.647818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0e5476f5-75c0-4efb-ac96-c7e45d971f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:00.657327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:00.750902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:01.043619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86426052-da36-4f6d-b21c-fcdb68153607 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:01.045266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-86426052-da36-4f6d-b21c-fcdb68153607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:01.158605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86426052-da36-4f6d-b21c-fcdb68153607 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:01.221010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:01.240737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749930-fd39-4017-a609-e77b3b86499d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:01.241356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749930-fd39-4017-a609-e77b3b86499d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:01.241628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749930-fd39-4017-a609-e77b3b86499d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:01.241893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749930-fd39-4017-a609-e77b3b86499d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:01.243533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-88749930-fd39-4017-a609-e77b3b86499d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:01.369407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749930-fd39-4017-a609-e77b3b86499d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:01.370759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749930-fd39-4017-a609-e77b3b86499d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:01.390034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 725/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:03.591505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948398-02c6-411d-a4bd-c9edf2e48ffa None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:03.592253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948398-02c6-411d-a4bd-c9edf2e48ffa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:03.592253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81948398-02c6-411d-a4bd-c9edf2e48ffa None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:03.593447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:03.599727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a18c5-a51b-4e15-893a-3c8a40a47422 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:03.600326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a18c5-a51b-4e15-893a-3c8a40a47422 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:03.600326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a18c5-a51b-4e15-893a-3c8a40a47422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:03.796591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fd4a18c5-a51b-4e15-893a-3c8a40a47422 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:03.797799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fd4a18c5-a51b-4e15-893a-3c8a40a47422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:03.837976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:03.899863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:04.192254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81948398-02c6-411d-a4bd-c9edf2e48ffa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:04.193900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-81948398-02c6-411d-a4bd-c9edf2e48ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:04.310159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81948398-02c6-411d-a4bd-c9edf2e48ffa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:04.352400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:04.393167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b7104-bfd2-444b-b650-7f9f14f26887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:04.393778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7104-bfd2-444b-b650-7f9f14f26887 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:04.394045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7104-bfd2-444b-b650-7f9f14f26887 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:04.394311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7104-bfd2-444b-b650-7f9f14f26887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:04.395915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b77b7104-bfd2-444b-b650-7f9f14f26887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:04.509136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b7104-bfd2-444b-b650-7f9f14f26887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:04.510505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b7104-bfd2-444b-b650-7f9f14f26887 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:04.529788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 738/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:06.713439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9309e3b-f2e5-4cd4-9a41-9a5741e917fb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:06.714020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9309e3b-f2e5-4cd4-9a41-9a5741e917fb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:06.714020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9309e3b-f2e5-4cd4-9a41-9a5741e917fb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:06.715318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:06.721007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e116c1-0198-4fe1-9d7b-f1d405674af6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:06.721508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e116c1-0198-4fe1-9d7b-f1d405674af6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:06.721508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e116c1-0198-4fe1-9d7b-f1d405674af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:06.792992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f9e116c1-0198-4fe1-9d7b-f1d405674af6 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:06.794232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f9e116c1-0198-4fe1-9d7b-f1d405674af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:06.813167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:06.897420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:07.177479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9309e3b-f2e5-4cd4-9a41-9a5741e917fb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:07.179029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a9309e3b-f2e5-4cd4-9a41-9a5741e917fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:07.297422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9309e3b-f2e5-4cd4-9a41-9a5741e917fb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:07.375746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:07.383090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d93326-d683-4ab7-af34-924efe8192b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:07.383708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93326-d683-4ab7-af34-924efe8192b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:07.383979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93326-d683-4ab7-af34-924efe8192b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:07.384244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d93326-d683-4ab7-af34-924efe8192b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:07.385907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-62d93326-d683-4ab7-af34-924efe8192b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:07.501369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93326-d683-4ab7-af34-924efe8192b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:07.502739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d93326-d683-4ab7-af34-924efe8192b2 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:07.522812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 728/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:09.675009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580f893-f7ff-4ce5-9ee0-01752add0dfe None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:09.675618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580f893-f7ff-4ce5-9ee0-01752add0dfe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:09.675618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580f893-f7ff-4ce5-9ee0-01752add0dfe None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:09.677005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 17:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:09.682714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127bc71-f54c-4768-a324-2dfb763af780 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:09.683599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127bc71-f54c-4768-a324-2dfb763af780 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:09.683874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127bc71-f54c-4768-a324-2dfb763af780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:09.752850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2127bc71-f54c-4768-a324-2dfb763af780 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:09.754102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2127bc71-f54c-4768-a324-2dfb763af780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:09.806583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:09.856612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 17:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:10.135528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580f893-f7ff-4ce5-9ee0-01752add0dfe None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:10.137399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1580f893-f7ff-4ce5-9ee0-01752add0dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:10.261514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580f893-f7ff-4ce5-9ee0-01752add0dfe None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:10.315317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:10.343389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e015505-3606-40b8-8007-81277711ed42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:10.344026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e015505-3606-40b8-8007-81277711ed42 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:10.344307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e015505-3606-40b8-8007-81277711ed42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:10.344600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e015505-3606-40b8-8007-81277711ed42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:10.346550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0e015505-3606-40b8-8007-81277711ed42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:10.478414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e015505-3606-40b8-8007-81277711ed42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:10.479766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e015505-3606-40b8-8007-81277711ed42 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:10.500969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 741/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:19.854941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=22 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:20.496281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=66 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:30.562237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7d8d9-c4d6-4033-bac4-a1eb632026ad None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:30.563349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7d8d9-c4d6-4033-bac4-a1eb632026ad None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:30.563349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7d8d9-c4d6-4033-bac4-a1eb632026ad None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:30.564395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 730/1471] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:30.571423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd6b4-7f10-4e29-aee5-b8d6abcf4df2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:30.571942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd6b4-7f10-4e29-aee5-b8d6abcf4df2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:30.571942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd6b4-7f10-4e29-aee5-b8d6abcf4df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:30.640718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-af5fd6b4-7f10-4e29-aee5-b8d6abcf4df2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:30.641853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-af5fd6b4-7f10-4e29-aee5-b8d6abcf4df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:30.750229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 742/1472] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:31.063114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97859d8-c83c-4899-a6e1-4ffedc6e34c8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:31.063393 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97859d8-c83c-4899-a6e1-4ffedc6e34c8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:31.063684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97859d8-c83c-4899-a6e1-4ffedc6e34c8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:31.065100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 731/1473] 10.222.0.43 () {54 vars in 966 bytes} [Tue Sep 13 17:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:31.070308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ef8ef-3ecc-49de-9e38-6fdb7408e4c5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:31.070596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ef8ef-3ecc-49de-9e38-6fdb7408e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:31.070900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ef8ef-3ecc-49de-9e38-6fdb7408e4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:31.157661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7f6ef8ef-3ecc-49de-9e38-6fdb7408e4c5 None None] MFA Rules not processed for user `2c282d6f237542e3b873100748759794`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:31.158903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7f6ef8ef-3ecc-49de-9e38-6fdb7408e4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:31.181495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:31.242273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 743/1474] 10.222.0.43 () {58 vars in 1063 bytes} [Tue Sep 13 17:56:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:31.250134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97859d8-c83c-4899-a6e1-4ffedc6e34c8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:31.251784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c97859d8-c83c-4899-a6e1-4ffedc6e34c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:31.360699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97859d8-c83c-4899-a6e1-4ffedc6e34c8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:31.435391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160b165-0fae-40ea-9c41-b527760d8d1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:31.436006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160b165-0fae-40ea-9c41-b527760d8d1f service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:31.436280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160b165-0fae-40ea-9c41-b527760d8d1f service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:31.436577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160b165-0fae-40ea-9c41-b527760d8d1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:31.438217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3160b165-0fae-40ea-9c41-b527760d8d1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:31.495774 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:31.552929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160b165-0fae-40ea-9c41-b527760d8d1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:31.554402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160b165-0fae-40ea-9c41-b527760d8d1f service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:31.574235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 732/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:33.804476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628859-2473-45f8-8f81-cac48df0a205 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:33.805078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628859-2473-45f8-8f81-cac48df0a205 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:33.805078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628859-2473-45f8-8f81-cac48df0a205 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:33.806372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 744/1476] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:33.812037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e42be6-650d-4574-9c91-e8fae2360d2b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:33.812531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e42be6-650d-4574-9c91-e8fae2360d2b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:33.812631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e42be6-650d-4574-9c91-e8fae2360d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:33.881675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e2e42be6-650d-4574-9c91-e8fae2360d2b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:33.882896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2e42be6-650d-4574-9c91-e8fae2360d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:33.981632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 733/1477] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:34.248854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628859-2473-45f8-8f81-cac48df0a205 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:34.250569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0a628859-2473-45f8-8f81-cac48df0a205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:34.502381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628859-2473-45f8-8f81-cac48df0a205 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:34.553551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:34.602183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0af03-2111-4af1-9f9d-af867ccb5d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:34.602786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0af03-2111-4af1-9f9d-af867ccb5d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:34.603064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0af03-2111-4af1-9f9d-af867ccb5d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:34.603329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0af03-2111-4af1-9f9d-af867ccb5d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:34.603968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0af03-2111-4af1-9f9d-af867ccb5d62 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:34.605764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-e3c0af03-2111-4af1-9f9d-af867ccb5d62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:56:34.605764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 17:56:34.605764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 17:56:34.606904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0af03-2111-4af1-9f9d-af867ccb5d62 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:34.626196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 745/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Tue Sep 13 17:56:34 2022] GET /identity/v3/projects => generated 2553 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:56:36.784208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ac12-f06c-4205-92cf-8aba33c41f3a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:36.784807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ac12-f06c-4205-92cf-8aba33c41f3a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:36.784807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ac12-f06c-4205-92cf-8aba33c41f3a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:36.786417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 734/1479] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:36.792229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f0088-616b-4562-a748-ca5c99add749 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:36.792709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f0088-616b-4562-a748-ca5c99add749 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:36.792920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f0088-616b-4562-a748-ca5c99add749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:36.879509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-410f0088-616b-4562-a748-ca5c99add749 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:36.881334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-410f0088-616b-4562-a748-ca5c99add749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:36.983199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 746/1480] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:37.277441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ac12-f06c-4205-92cf-8aba33c41f3a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:37.279071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d2b5ac12-f06c-4205-92cf-8aba33c41f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:37.290098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:37.430983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ac12-f06c-4205-92cf-8aba33c41f3a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:37.519706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:37.554156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca67ad-e79d-4aa5-8f98-9b83835fbfc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:37.554767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca67ad-e79d-4aa5-8f98-9b83835fbfc7 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:37.555056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca67ad-e79d-4aa5-8f98-9b83835fbfc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:37.555312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca67ad-e79d-4aa5-8f98-9b83835fbfc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:37.556977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8bca67ad-e79d-4aa5-8f98-9b83835fbfc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:37.651856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca67ad-e79d-4aa5-8f98-9b83835fbfc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:37.653566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca67ad-e79d-4aa5-8f98-9b83835fbfc7 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:37.673626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 735/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:40.031969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb133ae1-bd1a-4698-a899-6c36627823b7 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:40.032594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb133ae1-bd1a-4698-a899-6c36627823b7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:40.032594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb133ae1-bd1a-4698-a899-6c36627823b7 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:40.033899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 747/1482] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:40.039455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e102d-04cd-4678-bb81-c46e7dbf76ca None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:40.039904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e102d-04cd-4678-bb81-c46e7dbf76ca None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:40.040020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e102d-04cd-4678-bb81-c46e7dbf76ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:40.108601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-346e102d-04cd-4678-bb81-c46e7dbf76ca None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:40.109800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-346e102d-04cd-4678-bb81-c46e7dbf76ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:40.158929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:40.212388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 736/1483] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:40.495812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb133ae1-bd1a-4698-a899-6c36627823b7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:40.497428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fb133ae1-bd1a-4698-a899-6c36627823b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:40.524462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:40.609350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb133ae1-bd1a-4698-a899-6c36627823b7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:40.693855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bf379-508c-419b-9879-3c8caec40fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:40.694371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf379-508c-419b-9879-3c8caec40fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:40.694642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf379-508c-419b-9879-3c8caec40fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:40.694911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bf379-508c-419b-9879-3c8caec40fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:40.696553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9c4bf379-508c-419b-9879-3c8caec40fa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:40.777093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:40.815781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bf379-508c-419b-9879-3c8caec40fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:40.817267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bf379-508c-419b-9879-3c8caec40fa2 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:40.835566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 748/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:43.046550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9edada-c17f-492f-be95-68a66ad0b014 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:43.047291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9edada-c17f-492f-be95-68a66ad0b014 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:43.047291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9edada-c17f-492f-be95-68a66ad0b014 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:43.048591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 737/1485] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:43.054360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cac4-6d88-484e-91f9-5889e9e8b78b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:43.054844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cac4-6d88-484e-91f9-5889e9e8b78b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:43.054980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cac4-6d88-484e-91f9-5889e9e8b78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:43.126519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d373cac4-6d88-484e-91f9-5889e9e8b78b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:43.127752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d373cac4-6d88-484e-91f9-5889e9e8b78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:43.230564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 749/1486] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:43.513872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9edada-c17f-492f-be95-68a66ad0b014 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:43.515474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0e9edada-c17f-492f-be95-68a66ad0b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:43.625995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9edada-c17f-492f-be95-68a66ad0b014 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:43.702672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:43.704195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e825264-4672-43b9-95f0-c4400ac10833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:43.704829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e825264-4672-43b9-95f0-c4400ac10833 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:43.705102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e825264-4672-43b9-95f0-c4400ac10833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:43.705395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e825264-4672-43b9-95f0-c4400ac10833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:43.707492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9e825264-4672-43b9-95f0-c4400ac10833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:43.832834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e825264-4672-43b9-95f0-c4400ac10833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:43.834286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e825264-4672-43b9-95f0-c4400ac10833 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:43.853289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 738/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:46.139570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0bf8d-f6cc-4b44-ad5d-15ca7bb8d03c None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:46.140241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0bf8d-f6cc-4b44-ad5d-15ca7bb8d03c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:46.140241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0bf8d-f6cc-4b44-ad5d-15ca7bb8d03c None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:46.141815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 750/1488] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:46.149029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c719502-97ff-49e7-abff-cd84b3cb0faa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:46.149610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c719502-97ff-49e7-abff-cd84b3cb0faa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:46.149610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c719502-97ff-49e7-abff-cd84b3cb0faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:46.221082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8c719502-97ff-49e7-abff-cd84b3cb0faa None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:46.222316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8c719502-97ff-49e7-abff-cd84b3cb0faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:46.278835 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:46.324497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 739/1489] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:46.629228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0bf8d-f6cc-4b44-ad5d-15ca7bb8d03c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:46.631162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7ef0bf8d-f6cc-4b44-ad5d-15ca7bb8d03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:46.668537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:46.740055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0bf8d-f6cc-4b44-ad5d-15ca7bb8d03c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:46.824595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac923e-e3e3-4545-a346-b10fe7d040ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:46.825432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac923e-e3e3-4545-a346-b10fe7d040ea service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:46.825715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac923e-e3e3-4545-a346-b10fe7d040ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:46.825985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac923e-e3e3-4545-a346-b10fe7d040ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:46.827602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9fac923e-e3e3-4545-a346-b10fe7d040ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:46.912817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:46.942772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac923e-e3e3-4545-a346-b10fe7d040ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:46.944225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac923e-e3e3-4545-a346-b10fe7d040ea service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:46.962796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 751/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:49.418183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af33cf-00a9-4ee5-af9a-65541ad2c7b6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:49.418790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af33cf-00a9-4ee5-af9a-65541ad2c7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:49.418790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af33cf-00a9-4ee5-af9a-65541ad2c7b6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:49.420158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 740/1491] 10.222.0.43 () {54 vars in 927 bytes} [Tue Sep 13 17:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:56:49.425948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636d507-bd8b-4e26-962c-d51233391579 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:49.426426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636d507-bd8b-4e26-962c-d51233391579 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:49.426520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636d507-bd8b-4e26-962c-d51233391579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:49.496834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6636d507-bd8b-4e26-962c-d51233391579 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:56:49.498095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6636d507-bd8b-4e26-962c-d51233391579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:49.601169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 752/1492] 10.222.0.43 () {58 vars in 1024 bytes} [Tue Sep 13 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:56:49.887151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af33cf-00a9-4ee5-af9a-65541ad2c7b6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:56:49.888767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-73af33cf-00a9-4ee5-af9a-65541ad2c7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:49.995484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af33cf-00a9-4ee5-af9a-65541ad2c7b6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:56:50.071752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc72aa-ed2c-499f-9bbe-53478dafc4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:56:50.072363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc72aa-ed2c-499f-9bbe-53478dafc4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:56:50.072670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc72aa-ed2c-499f-9bbe-53478dafc4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:56:50.072933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc72aa-ed2c-499f-9bbe-53478dafc4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:56:50.074540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-80cc72aa-ed2c-499f-9bbe-53478dafc4e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:56:50.083555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:56:50.189807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc72aa-ed2c-499f-9bbe-53478dafc4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:56:50.191158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc72aa-ed2c-499f-9bbe-53478dafc4e5 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:56:50.210582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 741/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:56:59.599473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=82 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:57:00.208099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=50 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:09.979320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:09.979935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:09.979935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:10.068995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:58:10.070245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:10.181017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 753/1494] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 17:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:58:10.189579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f1a4c-3d28-4db0-9bc1-601d7c294298 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:10.190162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f1a4c-3d28-4db0-9bc1-601d7c294298 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:10.190162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f1a4c-3d28-4db0-9bc1-601d7c294298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:10.277002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-657f1a4c-3d28-4db0-9bc1-601d7c294298 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:58:10.278244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-657f1a4c-3d28-4db0-9bc1-601d7c294298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:10.384308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 742/1495] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:58:10.390937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:10.392575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:10.408245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ca684-4eed-4e89-a6db-df87c38fda6e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:10.474501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:10.512557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:10.513248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:10.513567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:10.514079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:10.514953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 17:58:10.515301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:10.516579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:10.516579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 17:58:10.516579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 17:58:10.517607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:10.530858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 754/1496] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 17:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 17:58:10.539835 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f1a4c-3d28-4db0-9bc1-601d7c294298 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:10.549678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f1a4c-3d28-4db0-9bc1-601d7c294298 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:10.625980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:10.668027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:10.668585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:10.668867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:10.669139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:10.669842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:10.671688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:10.671688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Sep 13 17:58:10.671688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: Sep 13 17:58:10.672922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:10.739298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 743/1497] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 17:58:10 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:58:10.744618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:10.746633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:10.790517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6a40c-2c5d-47b5-a2b4-f25d8f882779 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:10.916893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:10.917945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:10.918502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:10.918905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:10.919975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:10.921914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:10.921914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 17:58:10.921914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 17:58:10.924306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:10.986335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 755/1498] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 17:58:10 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 17:58:10.992132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:10.993691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:11.059933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036c07e-6d26-41a3-8bfe-16d76830ff1d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:11.151516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:11.163658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:11.164269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:11.164651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:11.164918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:11.165556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:11.166618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:11.166618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Sep 13 17:58:11.166618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: Sep 13 17:58:11.167515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:11.180702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 744/1499] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 17:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:58:11.185391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:11.187172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:11.200208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb5f85-ed73-4023-98f9-f472c90e095a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:11.228468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:11.303725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0c4e6-c3bf-4f6c-a7a3-0b37cadfce57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:11.304334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c4e6-c3bf-4f6c-a7a3-0b37cadfce57 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:11.304636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c4e6-c3bf-4f6c-a7a3-0b37cadfce57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:11.304899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c4e6-c3bf-4f6c-a7a3-0b37cadfce57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:11.305508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0c4e6-c3bf-4f6c-a7a3-0b37cadfce57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:11.306552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-79d0c4e6-c3bf-4f6c-a7a3-0b37cadfce57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:11.306552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 17:58:11.306552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 17:58:11.307427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0c4e6-c3bf-4f6c-a7a3-0b37cadfce57 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:11.321619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 756/1500] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 17:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 17:58:11.326259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:11.327995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:11.363485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8291af-30be-4521-acaf-7223c1d546a1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:11.465492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74349257-f5a6-4ba8-9da1-1cb5d68d3d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:11.466068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349257-f5a6-4ba8-9da1-1cb5d68d3d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:11.466336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349257-f5a6-4ba8-9da1-1cb5d68d3d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:11.466604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349257-f5a6-4ba8-9da1-1cb5d68d3d56 admin admin] PATH_INFO: `/v3/projects/8f6ad48aa423457d8ecdf4b0d718ac4c/users/108f03b88eac49a2978384274febfa99/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:11.490006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74349257-f5a6-4ba8-9da1-1cb5d68d3d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6ad48aa423457d8ecdf4b0d718ac4c, user_id=108f03b88eac49a2978384274febfa99, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:11.491043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-74349257-f5a6-4ba8-9da1-1cb5d68d3d56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:11.491043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Sep 13 17:58:11.491043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: Sep 13 17:58:11.493064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74349257-f5a6-4ba8-9da1-1cb5d68d3d56 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:11.524334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 745/1501] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 17:58:11 2022] PUT /identity/v3/projects/8f6ad48aa423457d8ecdf4b0d718ac4c/users/108f03b88eac49a2978384274febfa99/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:58:11.537643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf642a-f570-4067-9100-d07136c1f404 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:11.537936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf642a-f570-4067-9100-d07136c1f404 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:11.538214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf642a-f570-4067-9100-d07136c1f404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:11.600263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8edf642a-f570-4067-9100-d07136c1f404 None None] MFA Rules not processed for user `108f03b88eac49a2978384274febfa99`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:58:11.601508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8edf642a-f570-4067-9100-d07136c1f404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:11.701729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 757/1502] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:58:11.716212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6cf2e-7e7d-4948-9a80-8451afd63e85 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:11.716406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6cf2e-7e7d-4948-9a80-8451afd63e85 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:11.716709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6cf2e-7e7d-4948-9a80-8451afd63e85 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:11.718246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 746/1503] 10.222.0.43 () {54 vars in 964 bytes} [Tue Sep 13 17:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:58:11.723857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5679b-0a44-48d8-9e2f-926a19bcfb76 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:11.724325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5679b-0a44-48d8-9e2f-926a19bcfb76 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:11.724654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5679b-0a44-48d8-9e2f-926a19bcfb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:11.759419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:11.809647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a7c5679b-0a44-48d8-9e2f-926a19bcfb76 None None] MFA Rules not processed for user `6c9c5205f9344e8c8fcc85c80d6f504b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:58:11.810887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a7c5679b-0a44-48d8-9e2f-926a19bcfb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:11.893358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 758/1504] 10.222.0.43 () {58 vars in 1061 bytes} [Tue Sep 13 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:58:11.903278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6cf2e-7e7d-4948-9a80-8451afd63e85 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:11.904890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7aa6cf2e-7e7d-4948-9a80-8451afd63e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:11.958213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:12.025755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6cf2e-7e7d-4948-9a80-8451afd63e85 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:12.111897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:12.112538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:12.112814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:12.113078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:12.114693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:12.229174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:12.230854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:12.234706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:12.261134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 747/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:58:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:58:12.315359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f80d8-7a67-4119-b8fc-8f7bf36ae29e None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:12.315926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f80d8-7a67-4119-b8fc-8f7bf36ae29e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:12.316417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f80d8-7a67-4119-b8fc-8f7bf36ae29e None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:12.318237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 759/1506] 10.222.0.43 () {52 vars in 830 bytes} [Tue Sep 13 17:58:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:58:12.322477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:12.323702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:12.430592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eee789-a8aa-4f7e-a9fb-183b1add23ee service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:12.532374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:12.534056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:12.534056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:12.534056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] PATH_INFO: `/v3/users/108f03b88eac49a2978384274febfa99` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:12.549626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108f03b88eac49a2978384274febfa99)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:12.550595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:12.550595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Sep 13 17:58:12.550595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: Sep 13 17:58:12.551823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:12.563060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:12.634356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108f03b88eac49a2978384274febfa99'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:12.647046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108f03b88eac49a2978384274febfa99'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:12.647576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108f03b88eac49a2978384274febfa99'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:12.665651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108f03b88eac49a2978384274febfa99'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:12.698434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108f03b88eac49a2978384274febfa99'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:12.705635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 748/1507] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 17:58:12 2022] DELETE /identity/v3/users/108f03b88eac49a2978384274febfa99 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:58:12.710279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:12.710561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:12.710830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:12.774284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:58:12.775518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:12.857408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:12.871688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 760/1508] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 17:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:58:12.882193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:12.883485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:12.995422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf094f1-6338-49b5-8b12-c95b5dd06b31 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:13.061940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:13.083537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aae05b-f59b-43c3-ae7c-e65879b0c1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:13.084155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aae05b-f59b-43c3-ae7c-e65879b0c1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:13.084446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aae05b-f59b-43c3-ae7c-e65879b0c1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:13.084715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aae05b-f59b-43c3-ae7c-e65879b0c1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:13.086364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-23aae05b-f59b-43c3-ae7c-e65879b0c1f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:13.202017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aae05b-f59b-43c3-ae7c-e65879b0c1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:13.203394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aae05b-f59b-43c3-ae7c-e65879b0c1f4 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:13.234235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 749/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:58:13.613077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:58:13.614324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:58:13.626414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00927b-1389-4b2c-b80c-d0ea5d1d5544 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:58:13.752388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:58:13.753152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:58:13.753459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:58:13.753728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] PATH_INFO: `/v3/projects/8f6ad48aa423457d8ecdf4b0d718ac4c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:58:13.770358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6ad48aa423457d8ecdf4b0d718ac4c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:58:13.771404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 17:58:13.771404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 17:58:13.771404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 17:58:13.772655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:58:13.838103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6ad48aa423457d8ecdf4b0d718ac4c', 'request_id': 'req-78e6b3ca-8a7c-4535-b161-2919f1f1c060'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:13.850305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6ad48aa423457d8ecdf4b0d718ac4c', 'request_id': 'req-78e6b3ca-8a7c-4535-b161-2919f1f1c060'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:13.857130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6ad48aa423457d8ecdf4b0d718ac4c', 'request_id': 'req-78e6b3ca-8a7c-4535-b161-2919f1f1c060'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:13.892213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] The token cache is being invalidate because project 8f6ad48aa423457d8ecdf4b0d718ac4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 17:58:13.892570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:13.893000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-78e6b3ca-8a7c-4535-b161-2919f1f1c060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 17:58:13.894438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 761/1510] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 17:58:13 2022] DELETE /identity/v3/projects/8f6ad48aa423457d8ecdf4b0d718ac4c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 17:58:23.230047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=62 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:58:23.890446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:59:43.517567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d914-30fc-4f45-8c56-8a59a7342010 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:43.518242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d914-30fc-4f45-8c56-8a59a7342010 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:43.518242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5d914-30fc-4f45-8c56-8a59a7342010 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:43.519464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:59:43.525309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a5ca9-51b4-4dd9-97e0-44213562e604 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:43.525787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a5ca9-51b4-4dd9-97e0-44213562e604 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:43.525904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a5ca9-51b4-4dd9-97e0-44213562e604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:43.616118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4a8a5ca9-51b4-4dd9-97e0-44213562e604 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:59:43.617434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4a8a5ca9-51b4-4dd9-97e0-44213562e604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:59:43.728569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:59:43.887518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5d914-30fc-4f45-8c56-8a59a7342010 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:59:43.889734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8cb5d914-30fc-4f45-8c56-8a59a7342010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:59:44.023944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5d914-30fc-4f45-8c56-8a59a7342010 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:59:44.030951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512e574-c542-49ab-83f0-0a963333a1d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:59:44.031564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512e574-c542-49ab-83f0-0a963333a1d7 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:44.031836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512e574-c542-49ab-83f0-0a963333a1d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:44.032103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512e574-c542-49ab-83f0-0a963333a1d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:44.041698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512e574-c542-49ab-83f0-0a963333a1d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:59:44.043308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512e574-c542-49ab-83f0-0a963333a1d7 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:59:44.052881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 751/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:59:45.145606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f0bb-e3ca-426f-ae24-6c818fa3bc99 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:45.146291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f0bb-e3ca-426f-ae24-6c818fa3bc99 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:45.146291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f0bb-e3ca-426f-ae24-6c818fa3bc99 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:45.147524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:59:45.156686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c30a5e-e9cd-4736-82c2-bea3a1687f9d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:45.157492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c30a5e-e9cd-4736-82c2-bea3a1687f9d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:45.157492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c30a5e-e9cd-4736-82c2-bea3a1687f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:45.172981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882d4b7-8c81-435c-8b86-f4ea3f38aa7f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:45.173246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882d4b7-8c81-435c-8b86-f4ea3f38aa7f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:45.173576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882d4b7-8c81-435c-8b86-f4ea3f38aa7f None None] PATH_INFO: `/v3` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:45.174889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 764/1515] 10.222.0.65 () {54 vars in 925 bytes} [Tue Sep 13 17:59:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 17:59:45.177432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a6c30a5e-e9cd-4736-82c2-bea3a1687f9d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:59:45.178645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a6c30a5e-e9cd-4736-82c2-bea3a1687f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:59:45.181194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba389f9c-1ceb-43ed-94f8-5013e7d58d75 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:45.181476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba389f9c-1ceb-43ed-94f8-5013e7d58d75 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:45.181747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba389f9c-1ceb-43ed-94f8-5013e7d58d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:45.186973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 752/1516] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:59:45.242837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ba389f9c-1ceb-43ed-94f8-5013e7d58d75 None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:59:45.244050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ba389f9c-1ceb-43ed-94f8-5013e7d58d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:59:45.277490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 17:59:45.321734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 765/1517] 10.222.0.65 () {58 vars in 1015 bytes} [Tue Sep 13 17:59:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:59:45.337278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c30a5e-e9cd-4736-82c2-bea3a1687f9d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:59:45.355471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c30a5e-e9cd-4736-82c2-bea3a1687f9d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:59:45.362253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebff93-32b8-480a-806c-8ae2af95413e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:59:45.362869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebff93-32b8-480a-806c-8ae2af95413e service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:45.363147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebff93-32b8-480a-806c-8ae2af95413e service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:45.363421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebff93-32b8-480a-806c-8ae2af95413e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:45.372668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebff93-32b8-480a-806c-8ae2af95413e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:59:45.374283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebff93-32b8-480a-806c-8ae2af95413e service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:59:45.386482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 753/1518] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:59:48.848362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f09f0-94c6-42b3-b96d-bc7237892f15 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:48.848905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f09f0-94c6-42b3-b96d-bc7237892f15 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:48.849008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f09f0-94c6-42b3-b96d-bc7237892f15 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:48.850345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 766/1519] 10.222.0.43 () {54 vars in 961 bytes} [Tue Sep 13 17:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:59:48.855675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5ef9f-6940-41ba-a6e0-9ec79d4eea5a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:48.856117 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5ef9f-6940-41ba-a6e0-9ec79d4eea5a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:48.856219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5ef9f-6940-41ba-a6e0-9ec79d4eea5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:48.877093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-90b5ef9f-6940-41ba-a6e0-9ec79d4eea5a None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:59:48.878716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90b5ef9f-6940-41ba-a6e0-9ec79d4eea5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:59:48.888859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 754/1520] 10.222.0.43 () {58 vars in 1058 bytes} [Tue Sep 13 17:59:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:59:48.897149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f09f0-94c6-42b3-b96d-bc7237892f15 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:59:48.911681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f09f0-94c6-42b3-b96d-bc7237892f15 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:59:48.917128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99917fb-5bac-4a45-a9ba-e32ec03bc4b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:59:48.917721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99917fb-5bac-4a45-a9ba-e32ec03bc4b6 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:48.918009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99917fb-5bac-4a45-a9ba-e32ec03bc4b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:48.918265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99917fb-5bac-4a45-a9ba-e32ec03bc4b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:48.926952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99917fb-5bac-4a45-a9ba-e32ec03bc4b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:59:48.928619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99917fb-5bac-4a45-a9ba-e32ec03bc4b6 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:59:48.938888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 767/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:59:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:59:56.876916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73e59f-7757-4659-9cfc-2c5877b281ff None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:56.877514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73e59f-7757-4659-9cfc-2c5877b281ff None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:56.877514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73e59f-7757-4659-9cfc-2c5877b281ff None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:56.879858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:59:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:59:56.890533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c050d-76b3-4c70-82b7-65924eb6868d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:56.891030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c050d-76b3-4c70-82b7-65924eb6868d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:56.891123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c050d-76b3-4c70-82b7-65924eb6868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:56.915249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f82c050d-76b3-4c70-82b7-65924eb6868d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:59:56.916484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f82c050d-76b3-4c70-82b7-65924eb6868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:59:56.925071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:59:57.079634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73e59f-7757-4659-9cfc-2c5877b281ff None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:59:57.095194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73e59f-7757-4659-9cfc-2c5877b281ff None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:59:57.100959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54425f6-f550-4c3b-ae8c-0e217f01da3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:59:57.101560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54425f6-f550-4c3b-ae8c-0e217f01da3b service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:57.101837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54425f6-f550-4c3b-ae8c-0e217f01da3b service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:57.102138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54425f6-f550-4c3b-ae8c-0e217f01da3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:57.110612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54425f6-f550-4c3b-ae8c-0e217f01da3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:59:57.112217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54425f6-f550-4c3b-ae8c-0e217f01da3b service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:59:57.121873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 756/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 17:59:58.198035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ffaf-77ee-40b9-a483-fcba0ba9ef09 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:58.198674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ffaf-77ee-40b9-a483-fcba0ba9ef09 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:58.198674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ffaf-77ee-40b9-a483-fcba0ba9ef09 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:58.199964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 17:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 17:59:58.206785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ba62-7fde-4d77-8f3d-64c5838de86f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:58.207272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ba62-7fde-4d77-8f3d-64c5838de86f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:58.207381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ba62-7fde-4d77-8f3d-64c5838de86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:58.229089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8100ba62-7fde-4d77-8f3d-64c5838de86f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 17:59:58.230295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8100ba62-7fde-4d77-8f3d-64c5838de86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 17:59:58.238831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 17:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 17:59:58.391752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8ffaf-77ee-40b9-a483-fcba0ba9ef09 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 17:59:58.407473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8ffaf-77ee-40b9-a483-fcba0ba9ef09 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 17:59:58.413297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda05b1-bee7-478e-bc05-805628f0b801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 17:59:58.413898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda05b1-bee7-478e-bc05-805628f0b801 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 17:59:58.414175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda05b1-bee7-478e-bc05-805628f0b801 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 17:59:58.414449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda05b1-bee7-478e-bc05-805628f0b801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 17:59:58.423165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda05b1-bee7-478e-bc05-805628f0b801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 17:59:58.424684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda05b1-bee7-478e-bc05-805628f0b801 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 17:59:58.434591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 770/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 17:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:00:02.512280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ef45-736e-4a92-a39d-1bc74c5156fb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:02.512878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ef45-736e-4a92-a39d-1bc74c5156fb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:02.512878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ef45-736e-4a92-a39d-1bc74c5156fb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:02.514264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 18:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:00:02.519774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe080723-5914-4286-bdf2-122e00266a34 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:02.520199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe080723-5914-4286-bdf2-122e00266a34 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:02.520294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe080723-5914-4286-bdf2-122e00266a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:02.543459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fe080723-5914-4286-bdf2-122e00266a34 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:00:02.544715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fe080723-5914-4286-bdf2-122e00266a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:02.554460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 18:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:00:02.841390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13ef45-736e-4a92-a39d-1bc74c5156fb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:00:02.843440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3e13ef45-736e-4a92-a39d-1bc74c5156fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:02.911130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:00:02.929705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13ef45-736e-4a92-a39d-1bc74c5156fb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:00:02.933708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab6271-7c7f-4d9b-bfb2-2ee52684e502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:00:02.934318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab6271-7c7f-4d9b-bfb2-2ee52684e502 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:02.934596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab6271-7c7f-4d9b-bfb2-2ee52684e502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:02.934862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab6271-7c7f-4d9b-bfb2-2ee52684e502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:02.944415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab6271-7c7f-4d9b-bfb2-2ee52684e502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:00:02.945904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab6271-7c7f-4d9b-bfb2-2ee52684e502 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:00:02.955296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 759/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:00:05.072914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd75-1be8-4022-b507-4e05f1bc4043 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:05.073493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd75-1be8-4022-b507-4e05f1bc4043 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:05.073493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd75-1be8-4022-b507-4e05f1bc4043 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:05.074787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 18:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:00:05.081250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ae365-db5a-49b5-9252-489f92ddedfa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:05.081915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ae365-db5a-49b5-9252-489f92ddedfa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:05.082069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ae365-db5a-49b5-9252-489f92ddedfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:05.106616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-824ae365-db5a-49b5-9252-489f92ddedfa None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:00:05.107862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-824ae365-db5a-49b5-9252-489f92ddedfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:05.117187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:00:05.423538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74fd75-1be8-4022-b507-4e05f1bc4043 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:00:05.426755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4d74fd75-1be8-4022-b507-4e05f1bc4043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:05.447695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74fd75-1be8-4022-b507-4e05f1bc4043 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:00:05.451841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5617d5-c14d-4fc5-b3f7-b7385c15fc7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:00:05.452482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5617d5-c14d-4fc5-b3f7-b7385c15fc7e service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:05.452763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5617d5-c14d-4fc5-b3f7-b7385c15fc7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:05.453030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5617d5-c14d-4fc5-b3f7-b7385c15fc7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:05.464174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5617d5-c14d-4fc5-b3f7-b7385c15fc7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:00:05.465695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5617d5-c14d-4fc5-b3f7-b7385c15fc7e service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:00:05.476101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 773/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:00:09.243184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed8e23-adc0-4559-a0b2-c4842d1e71a9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:09.243798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed8e23-adc0-4559-a0b2-c4842d1e71a9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:09.243798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed8e23-adc0-4559-a0b2-c4842d1e71a9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:09.245672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 18:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 18:00:09.254714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aedd6-6263-4155-b1f7-0d07a544ab24 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:09.255213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aedd6-6263-4155-b1f7-0d07a544ab24 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:09.255213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aedd6-6263-4155-b1f7-0d07a544ab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:09.276271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-317aedd6-6263-4155-b1f7-0d07a544ab24 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:00:09.277510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-317aedd6-6263-4155-b1f7-0d07a544ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:09.286635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 18:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:00:09.585006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed8e23-adc0-4559-a0b2-c4842d1e71a9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:00:09.587852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72ed8e23-adc0-4559-a0b2-c4842d1e71a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:09.678705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed8e23-adc0-4559-a0b2-c4842d1e71a9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:00:09.683076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003e0df-8626-4194-b387-05d2a1776992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:00:09.683690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e0df-8626-4194-b387-05d2a1776992 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:09.683956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e0df-8626-4194-b387-05d2a1776992 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:09.684226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e0df-8626-4194-b387-05d2a1776992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:09.694069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003e0df-8626-4194-b387-05d2a1776992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:00:09.695518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003e0df-8626-4194-b387-05d2a1776992 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:00:09.705100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 762/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:00:11.883459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274769-2612-4830-b75d-664e6c05552d None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:11.884052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274769-2612-4830-b75d-664e6c05552d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:11.884052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc274769-2612-4830-b75d-664e6c05552d None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:11.885462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 18:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:00:11.892034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ef12e-efc9-4c1c-98c0-eeb3f3ebf3cf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:11.892856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ef12e-efc9-4c1c-98c0-eeb3f3ebf3cf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:11.893031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ef12e-efc9-4c1c-98c0-eeb3f3ebf3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:11.915786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-430ef12e-efc9-4c1c-98c0-eeb3f3ebf3cf None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:00:11.917051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-430ef12e-efc9-4c1c-98c0-eeb3f3ebf3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:11.925519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 18:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:00:12.221813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274769-2612-4830-b75d-664e6c05552d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:00:12.223939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dc274769-2612-4830-b75d-664e6c05552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:00:12.238542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc274769-2612-4830-b75d-664e6c05552d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:00:12.242661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621cb96-8250-43c8-9d0b-bb6a8436ac04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:00:12.243269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621cb96-8250-43c8-9d0b-bb6a8436ac04 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:00:12.243547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621cb96-8250-43c8-9d0b-bb6a8436ac04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:00:12.243816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621cb96-8250-43c8-9d0b-bb6a8436ac04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:00:12.254711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621cb96-8250-43c8-9d0b-bb6a8436ac04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:00:12.256148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621cb96-8250-43c8-9d0b-bb6a8436ac04 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:00:12.266220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 776/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:00:21.910858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=50 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:00:22.261879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=50 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:35.167849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ac47f-05c4-4caa-a93d-c2a1304864a1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.168589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ac47f-05c4-4caa-a93d-c2a1304864a1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.168897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ac47f-05c4-4caa-a93d-c2a1304864a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.194595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8b8ac47f-05c4-4caa-a93d-c2a1304864a1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.195817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8b8ac47f-05c4-4caa-a93d-c2a1304864a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.208644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 764/1540] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.217918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6094-ef0c-4af5-ae84-bf06e6148fb2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.218541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6094-ef0c-4af5-ae84-bf06e6148fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.218541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6094-ef0c-4af5-ae84-bf06e6148fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.256994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8bad6094-ef0c-4af5-ae84-bf06e6148fb2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.258866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8bad6094-ef0c-4af5-ae84-bf06e6148fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.274537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 777/1541] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.279056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ac47f-05c4-4caa-a93d-c2a1304864a1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.289078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ac47f-05c4-4caa-a93d-c2a1304864a1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.301250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.302018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.302291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.302556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.303912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:35.304211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.305289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 18:01:35.305289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Sep 13 18:01:35.305289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: Sep 13 18:01:35.306188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.323943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 765/1542] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:35.332076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6094-ef0c-4af5-ae84-bf06e6148fb2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.348491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6094-ef0c-4af5-ae84-bf06e6148fb2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.365430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656827a2-6c41-42a5-a2e6-ae80b6d1c98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.366083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656827a2-6c41-42a5-a2e6-ae80b6d1c98a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.366544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656827a2-6c41-42a5-a2e6-ae80b6d1c98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.366820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656827a2-6c41-42a5-a2e6-ae80b6d1c98a admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.367484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656827a2-6c41-42a5-a2e6-ae80b6d1c98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.368599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-656827a2-6c41-42a5-a2e6-ae80b6d1c98a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 18:01:35.368599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 18:01:35.368599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 18:01:35.370443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656827a2-6c41-42a5-a2e6-ae80b6d1c98a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.428484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 778/1543] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:35.432177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.443339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d9f8a-2048-48d3-ad52-f0794c47876d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.526207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dff77-7ea9-4bc5-a068-094c579f27cf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.526469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dff77-7ea9-4bc5-a068-094c579f27cf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.526746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dff77-7ea9-4bc5-a068-094c579f27cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.535689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.536451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.536733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.537001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.537657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.538765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 18:01:35.538765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Sep 13 18:01:35.538765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: Sep 13 18:01:35.540124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.577671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d59dff77-7ea9-4bc5-a068-094c579f27cf None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.578886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d59dff77-7ea9-4bc5-a068-094c579f27cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.579891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 766/1544] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:35.583930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.594569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cecb2a-84f6-44aa-ae57-97b524d2260f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.597355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 779/1545] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.604734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.605372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.605647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.605916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.606568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.607263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fe533-168e-4bfd-ae33-ae1dce4b5758 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.607551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fe533-168e-4bfd-ae33-ae1dce4b5758 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.607822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fe533-168e-4bfd-ae33-ae1dce4b5758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.608262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.624693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 767/1546] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:01:35.628975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.644313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf8d02-639a-49de-8aca-a97185bc9acc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.658090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baef032-c59d-4e39-81e6-18a0acc7f9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.658713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baef032-c59d-4e39-81e6-18a0acc7f9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.658984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baef032-c59d-4e39-81e6-18a0acc7f9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.659255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baef032-c59d-4e39-81e6-18a0acc7f9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.659901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baef032-c59d-4e39-81e6-18a0acc7f9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.661658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baef032-c59d-4e39-81e6-18a0acc7f9e1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.665555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-577fe533-168e-4bfd-ae33-ae1dce4b5758 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.666781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-577fe533-168e-4bfd-ae33-ae1dce4b5758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.677556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 768/1547] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:01:35.680607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963a81b-8178-4d8a-99df-887e2340e07a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.680607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963a81b-8178-4d8a-99df-887e2340e07a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.680607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963a81b-8178-4d8a-99df-887e2340e07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.682523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 780/1548] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.688457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fe533-168e-4bfd-ae33-ae1dce4b5758 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.708788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fe533-168e-4bfd-ae33-ae1dce4b5758 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.721440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1963a81b-8178-4d8a-99df-887e2340e07a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.722657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1963a81b-8178-4d8a-99df-887e2340e07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.724292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4b878-5a51-4fd4-87ba-f66c2b767779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.725556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b878-5a51-4fd4-87ba-f66c2b767779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.725833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b878-5a51-4fd4-87ba-f66c2b767779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.726099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b878-5a51-4fd4-87ba-f66c2b767779 admin admin] PATH_INFO: `/v3/projects/2eae4acecf6c4961830ead78d71b2ad3/users/60800cf2774e409d9c1b6f4992669991/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.749570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 769/1549] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.750434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963a81b-8178-4d8a-99df-887e2340e07a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.758753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4b878-5a51-4fd4-87ba-f66c2b767779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eae4acecf6c4961830ead78d71b2ad3, user_id=60800cf2774e409d9c1b6f4992669991, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.759800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-9fa4b878-5a51-4fd4-87ba-f66c2b767779 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 18:01:35.759800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 18:01:35.759800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 18:01:35.761778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4b878-5a51-4fd4-87ba-f66c2b767779 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.767822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963a81b-8178-4d8a-99df-887e2340e07a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.778174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c9aab-3932-4c5d-a2cf-bb478c99e5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.778806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c9aab-3932-4c5d-a2cf-bb478c99e5ed admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.779097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c9aab-3932-4c5d-a2cf-bb478c99e5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.779340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c9aab-3932-4c5d-a2cf-bb478c99e5ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.780037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c9aab-3932-4c5d-a2cf-bb478c99e5ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:35.780541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c9aab-3932-4c5d-a2cf-bb478c99e5ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.782536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c9aab-3932-4c5d-a2cf-bb478c99e5ed admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.787710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 781/1550] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:35 2022] PUT /identity/v3/projects/2eae4acecf6c4961830ead78d71b2ad3/users/60800cf2774e409d9c1b6f4992669991/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:35.789667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804f10-d73d-4895-922b-64d7bf63643b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.790346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804f10-d73d-4895-922b-64d7bf63643b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.790866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804f10-d73d-4895-922b-64d7bf63643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.811343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 770/1551] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:35.812672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7eb780-5baa-40c1-b7d8-6ab52ee09c6d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.812953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7eb780-5baa-40c1-b7d8-6ab52ee09c6d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.813251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7eb780-5baa-40c1-b7d8-6ab52ee09c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.814457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5f804f10-d73d-4895-922b-64d7bf63643b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.816293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5f804f10-d73d-4895-922b-64d7bf63643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.844027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0e7eb780-5baa-40c1-b7d8-6ab52ee09c6d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.845281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0e7eb780-5baa-40c1-b7d8-6ab52ee09c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.892254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 782/1552] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.893644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08327ce-6083-43b0-a140-4b73a0875314 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.893939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08327ce-6083-43b0-a140-4b73a0875314 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.894436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08327ce-6083-43b0-a140-4b73a0875314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.912969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 771/1553] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.914313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec374df-0862-4e7a-9549-a74c8a787299 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.914606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec374df-0862-4e7a-9549-a74c8a787299 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.914870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec374df-0862-4e7a-9549-a74c8a787299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.915651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d08327ce-6083-43b0-a140-4b73a0875314 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.916897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d08327ce-6083-43b0-a140-4b73a0875314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.925836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 783/1554] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.926705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08327ce-6083-43b0-a140-4b73a0875314 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:35.934910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0ec374df-0862-4e7a-9549-a74c8a787299 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.936022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0ec374df-0862-4e7a-9549-a74c8a787299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.936674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08327ce-6083-43b0-a140-4b73a0875314 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:35.941252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b3154-7d92-4ea2-9428-6efbe8f29d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:35.941849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3154-7d92-4ea2-9428-6efbe8f29d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.942123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3154-7d92-4ea2-9428-6efbe8f29d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.942426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b3154-7d92-4ea2-9428-6efbe8f29d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.943055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b3154-7d92-4ea2-9428-6efbe8f29d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:35.943896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 772/1555] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.945005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2290d8-13e1-44a1-a91f-3e283589bebd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.945128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b3154-7d92-4ea2-9428-6efbe8f29d18 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:35.945304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2290d8-13e1-44a1-a91f-3e283589bebd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.945553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2290d8-13e1-44a1-a91f-3e283589bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.965259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3a2290d8-13e1-44a1-a91f-3e283589bebd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.966420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3a2290d8-13e1-44a1-a91f-3e283589bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:35.974799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 773/1556] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:35.975057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 784/1557] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:35.976085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fdf8c-afe7-4d2c-8b28-7ce9cdafd074 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.976291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaba15d-e32d-46c1-9284-3b9f7c4fbaae None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:35.976410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fdf8c-afe7-4d2c-8b28-7ce9cdafd074 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.976650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaba15d-e32d-46c1-9284-3b9f7c4fbaae None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:35.976754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fdf8c-afe7-4d2c-8b28-7ce9cdafd074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.976924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaba15d-e32d-46c1-9284-3b9f7c4fbaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:35.998388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-035fdf8c-afe7-4d2c-8b28-7ce9cdafd074 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:35.999639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-035fdf8c-afe7-4d2c-8b28-7ce9cdafd074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:36.000171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4aaba15d-e32d-46c1-9284-3b9f7c4fbaae None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:36.001392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4aaba15d-e32d-46c1-9284-3b9f7c4fbaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:36.010064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:36.052184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:36.060876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 774/1558] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:36.068022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee77f-a25b-456f-a56b-2f9bf35cfe56 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.068022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee77f-a25b-456f-a56b-2f9bf35cfe56 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.068022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee77f-a25b-456f-a56b-2f9bf35cfe56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.068374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 785/1559] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:36.068374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaba15d-e32d-46c1-9284-3b9f7c4fbaae None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.075747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaba15d-e32d-46c1-9284-3b9f7c4fbaae None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.080836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.081438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.081717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.081983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.082628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:36.082925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.084611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.085943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-11fee77f-a25b-456f-a56b-2f9bf35cfe56 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:36.087145 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-11fee77f-a25b-456f-a56b-2f9bf35cfe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:36.095662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 775/1560] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:36.096740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 786/1561] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:36.097797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.099706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b19f4-e4c0-4e00-8495-69c18f902518 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.100217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b19f4-e4c0-4e00-8495-69c18f902518 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.100669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b19f4-e4c0-4e00-8495-69c18f902518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.120008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-648b19f4-e4c0-4e00-8495-69c18f902518 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:36.120964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-648b19f4-e4c0-4e00-8495-69c18f902518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:36.129508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 776/1562] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:36.130742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fa521-f090-4577-bb80-8f9aa61dd297 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.131020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fa521-f090-4577-bb80-8f9aa61dd297 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.131289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fa521-f090-4577-bb80-8f9aa61dd297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.150521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1b6fa521-f090-4577-bb80-8f9aa61dd297 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:36.151729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1b6fa521-f090-4577-bb80-8f9aa61dd297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:36.158741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868ba08-8182-41bd-ba6b-5377a716e9e8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.160992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 777/1563] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:36.161576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fa521-f090-4577-bb80-8f9aa61dd297 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.166209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059c694-f747-48c6-82f5-923cba4be9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.166827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c694-f747-48c6-82f5-923cba4be9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.167081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c694-f747-48c6-82f5-923cba4be9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.167383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059c694-f747-48c6-82f5-923cba4be9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.171805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fa521-f090-4577-bb80-8f9aa61dd297 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.177227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059c694-f747-48c6-82f5-923cba4be9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.177833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac6a56-d892-438e-9fba-ca0fc7a46f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.178434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6a56-d892-438e-9fba-ca0fc7a46f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.178704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6a56-d892-438e-9fba-ca0fc7a46f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.178813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059c694-f747-48c6-82f5-923cba4be9b4 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.178964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac6a56-d892-438e-9fba-ca0fc7a46f80 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.179590 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6a56-d892-438e-9fba-ca0fc7a46f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.181564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac6a56-d892-438e-9fba-ca0fc7a46f80 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.188915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 787/1564] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:36.190159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa7697-2d7a-4a4d-979c-b03f85192f58 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.190440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa7697-2d7a-4a4d-979c-b03f85192f58 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.190726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa7697-2d7a-4a4d-979c-b03f85192f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.210060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 778/1565] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.214879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9cfa7697-2d7a-4a4d-979c-b03f85192f58 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:36.215093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0692c-7ef7-4373-8251-be3d1e6443b2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.215354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0692c-7ef7-4373-8251-be3d1e6443b2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.215631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0692c-7ef7-4373-8251-be3d1e6443b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.216141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9cfa7697-2d7a-4a4d-979c-b03f85192f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:36.226897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 788/1566] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:36.227816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa7697-2d7a-4a4d-979c-b03f85192f58 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.237635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-62f0692c-7ef7-4373-8251-be3d1e6443b2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:36.246258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-62f0692c-7ef7-4373-8251-be3d1e6443b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:36.252088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa7697-2d7a-4a4d-979c-b03f85192f58 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.257965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.258663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.258998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.259335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.260093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:36.260488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.262294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.264051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 779/1567] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:36.264905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0692c-7ef7-4373-8251-be3d1e6443b2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.276314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 789/1568] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:36.277352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.278973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0692c-7ef7-4373-8251-be3d1e6443b2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.284336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.285240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.285637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.285959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.286887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.288088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ea507-eb8e-4532-b598-6d3121900929 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.289101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.293458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.294175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.294546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.294905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.295636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:36.296153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.297725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.311747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 790/1569] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:36.312888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.321388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 780/1570] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.322416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.324911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f8ebd-f453-45b8-8e20-100e8c3a1a90 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.396744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.397527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.397859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.398146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.398794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:36.399155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.402390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.402486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f58a84-73be-4e55-bd6f-47f16558343d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.416972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 791/1571] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:36.417997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.424747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.425951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.426350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.426744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.427527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.428157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df1d80-ce96-45c8-b88b-13ce3b8e81b9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.429852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.433095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.433754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.434030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.434296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.434940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:36.435232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.437445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.444128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 781/1572] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:36.445561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.450341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 792/1573] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:36.451228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.456779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef9096-d39c-434f-ba23-63607d9c192f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.461542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d786c-a55a-4a10-a7c9-5aef481c5640 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.461714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.462191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.462458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.462726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.463382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:36.463679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.465358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.466092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.466703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.466978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.467244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.467881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.469819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.485325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 782/1574] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:36.486430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.501579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b844e9-c056-491e-9c98-6d5866c14b2a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.507732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.507925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 793/1575] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.508763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.509029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.509621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.510065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.511451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.514842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.522874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:36.555189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 783/1576] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.555189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.571781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:36.584350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1edc-25c4-4976-a28a-8ff5b787c34a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.649143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62bca7-39a9-46e2-9fbd-4831a8e656de admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.653094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.654113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.654452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.655048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.655776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.658503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.659089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.659781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.660155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.660519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.661301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.663369 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.693048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 794/1577] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.694058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.707345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc23a2-b296-494b-a945-9d2f163c235f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.713781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 784/1578] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.713781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.716093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.716816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.717155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.717480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.718196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.720161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.733963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 795/1579] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:36.734865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.746379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf927-0055-41d7-9e70-4b7b98bff89f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.783650 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4385633-814b-4b01-9947-21420f715b10 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.789109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.789859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.790230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.790720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.791832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.793550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.810586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.811214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.811546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.811839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.812410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.814512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.829414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 785/1580] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/projects => generated 409 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.830580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.847178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 796/1581] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.848332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.885055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:36.911931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ec449-55b2-492b-90be-e298a394a271 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.944866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687f85c-169c-48f8-9ac9-2c12c612576f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:36.945456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:36.951061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.951687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.951969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.952444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.952979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.955114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:36.984216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 797/1582] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:36.987402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:36.987402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:36.987402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:36.987402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:36.987770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:36.988998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:36.989978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.004072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0c77-17d5-407f-b866-0e2e6cd4cd5e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.015027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.016813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.017253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.017888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.018776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.020409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 786/1583] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:36 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:37.021130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.021354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.032014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35da47-96b9-4831-8b3c-16f3776a3a0f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.037171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.037956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.038355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.038715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.039450 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.040085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 798/1584] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.041046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.041813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.051961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be620884-6d63-444b-b399-a25c9cbbd65b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.058721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.059437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.059757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.060014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] PATH_INFO: `/v3/projects/75e14fd88bfb4cd896fa7b070a0fac47/users/e68ecd9b3ca643539b109f75bb242d6f/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.076705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 787/1585] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:37 2022] POST /identity/v3/users => generated 286 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:37.079148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.080972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e14fd88bfb4cd896fa7b070a0fac47, user_id=e68ecd9b3ca643539b109f75bb242d6f, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.083714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.092912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7af1e-5dd7-4515-b40b-6e4ff276d615 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.099801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.101881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.102471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.102963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.104980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.110936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 799/1586] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:37 2022] PUT /identity/v3/projects/75e14fd88bfb4cd896fa7b070a0fac47/users/e68ecd9b3ca643539b109f75bb242d6f/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:37.110936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.111558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.159972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 788/1587] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:37 2022] POST /identity/v3/users => generated 305 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:37.161567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.180847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814fba4-6211-47c4-a4ca-bea19114a34d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.198763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828dc2e-d5ed-432e-9577-af399f77abbb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.203979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.204620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.204920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.205240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.205905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.208307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.239308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 800/1588] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:37 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:37.240039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.253456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefda2d3-e0cc-4552-91fa-e251ab6944b1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.259779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.260409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.260705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.261985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.262378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.264275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.270283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.271077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.271350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.271670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.272460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.274410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.278311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 801/1589] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.278715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.288593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 789/1590] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.289286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.291893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4af89-0645-45a0-9318-aea8ea43d072 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.297731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054b1a2-ef3e-4691-8328-8aad4356931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.298620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054b1a2-ef3e-4691-8328-8aad4356931d admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.299102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054b1a2-ef3e-4691-8328-8aad4356931d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.299567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054b1a2-ef3e-4691-8328-8aad4356931d admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.300617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054b1a2-ef3e-4691-8328-8aad4356931d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.301053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dada2-2bc2-4a73-9838-9a18ee7b0bcc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.302693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054b1a2-ef3e-4691-8328-8aad4356931d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.306797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.307160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.307806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.308138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.309165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.310607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.318891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 802/1591] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.320706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204b3ec-127d-4d9c-864f-f34cfc23b949 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.321245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204b3ec-127d-4d9c-864f-f34cfc23b949 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.321592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204b3ec-127d-4d9c-864f-f34cfc23b949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.325739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 790/1592] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.325739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.337152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f57eb9-4486-4c4a-9599-21a3fa9b2230 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.342012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.342621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.342891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.343154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.343800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.345734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.346553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4204b3ec-127d-4d9c-864f-f34cfc23b949 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:37.349246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4204b3ec-127d-4d9c-864f-f34cfc23b949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:37.362594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 803/1593] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:37.362833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 791/1594] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.362833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.363487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204b3ec-127d-4d9c-864f-f34cfc23b949 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.373499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204b3ec-127d-4d9c-864f-f34cfc23b949 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.379883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.380586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.380871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.381144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.381856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.382679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b26cca-626c-4b91-810b-ea78227a6076 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.383730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.387956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.388787 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.389294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.389736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.390538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.392301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.400694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 804/1595] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.400694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.408882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 792/1596] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.410063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.410222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369e406-30fc-4323-9fe5-8e976c0f5c18 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.415913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.416768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.417092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.417372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.418275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.420101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.423582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144a56b-2558-4753-b6fc-4c9c222a5754 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.428893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.429814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.430255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.430676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] PATH_INFO: `/v3/projects/a279fcbf274c4704a31a2b95061e1469/users/0625b3113ce044bbac4f6142eaade11c/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.433445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 805/1597] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.434175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.445939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47641-3df0-4434-883e-6333f0959832 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.451152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a279fcbf274c4704a31a2b95061e1469, user_id=0625b3113ce044bbac4f6142eaade11c, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.451972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.452699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.453058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.453403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.454186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.455497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.455966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.468017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 806/1598] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.469321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.475287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 793/1599] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:37 2022] PUT /identity/v3/projects/a279fcbf274c4704a31a2b95061e1469/users/0625b3113ce044bbac4f6142eaade11c/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:37.476764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.485278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7009ec-a452-41af-bfd1-6ae359027918 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.490740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:37.513153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:37.561288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b6fd-3c8b-451e-b364-708bbb881b4b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.568131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.569446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.569892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.570221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.594051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.595728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.596010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.596283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.597017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.600156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.611707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 807/1600] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.612643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.623211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d642b-7cd7-46b7-b5cb-1a265867fe34 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.628411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.629217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.629498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.629768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.630492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.632048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.643423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 808/1601] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:37.644281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.657310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9837c-3ce6-4df8-8faa-9ce7fbc7151c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.660012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.664099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.665437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.666455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.667101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.667803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] PATH_INFO: `/v3/projects/15fa8103e3a94cb8ac8fa73a29a1877d/users/c3c4dc5ead6445ce91b8625a9e170f36/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.681495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 794/1602] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:37.684833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.693387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15fa8103e3a94cb8ac8fa73a29a1877d, user_id=c3c4dc5ead6445ce91b8625a9e170f36, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.696778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.698312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72ac21-45ad-4ad0-a2e5-38ea79c8fa23 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.706368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b488b-202e-44d8-aa89-86fa2927db61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.707783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b488b-202e-44d8-aa89-86fa2927db61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.708402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b488b-202e-44d8-aa89-86fa2927db61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.709167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b488b-202e-44d8-aa89-86fa2927db61 admin admin] PATH_INFO: `/v3/projects/66136f123f5b4ad0a824920bbdf9d245/users/51db5ef052144879b53979268ac4ceb5/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.733003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 809/1603] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:37 2022] PUT /identity/v3/projects/15fa8103e3a94cb8ac8fa73a29a1877d/users/c3c4dc5ead6445ce91b8625a9e170f36/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:37.735610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.741270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b488b-202e-44d8-aa89-86fa2927db61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66136f123f5b4ad0a824920bbdf9d245, user_id=51db5ef052144879b53979268ac4ceb5, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.748612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b488b-202e-44d8-aa89-86fa2927db61 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.777893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 795/1604] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:37 2022] PUT /identity/v3/projects/66136f123f5b4ad0a824920bbdf9d245/users/51db5ef052144879b53979268ac4ceb5/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:37.781696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb7b7-f9eb-461d-a156-69041c1dce0d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.781696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb7b7-f9eb-461d-a156-69041c1dce0d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.781696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cb7b7-f9eb-461d-a156-69041c1dce0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.831342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a84cb7b7-f9eb-461d-a156-69041c1dce0d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:37.835476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83818e6b-2c9d-4a25-b3b0-c403dfd931d8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.836255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a84cb7b7-f9eb-461d-a156-69041c1dce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:37.952122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.952909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.953293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.953604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] PATH_INFO: `/v3/projects/e9c4a325015941f8b09c0e6c3ba50d63/users/5ba7e99def8f4744946e7b1697f55a44/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.967411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 796/1605] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:37.968922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb7b7-f9eb-461d-a156-69041c1dce0d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:37.974020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c4a325015941f8b09c0e6c3ba50d63, user_id=5ba7e99def8f4744946e7b1697f55a44, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:37.976921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:37.984018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84cb7b7-f9eb-461d-a156-69041c1dce0d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:37.990863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:37.992885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f9ae4-0525-4b09-9379-70e14c228127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:37.994014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f9ae4-0525-4b09-9379-70e14c228127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:37.994881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f9ae4-0525-4b09-9379-70e14c228127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:37.995694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f9ae4-0525-4b09-9379-70e14c228127 admin admin] PATH_INFO: `/v3/projects/4a9bc1ff5f234d1db581da768a84b07e/users/651bfff9429a4578963c9d6699962a1a/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:37.998005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 810/1606] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:37 2022] PUT /identity/v3/projects/e9c4a325015941f8b09c0e6c3ba50d63/users/5ba7e99def8f4744946e7b1697f55a44/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:37.998818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:38.020843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f9ae4-0525-4b09-9379-70e14c228127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9bc1ff5f234d1db581da768a84b07e, user_id=651bfff9429a4578963c9d6699962a1a, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:38.024063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f9ae4-0525-4b09-9379-70e14c228127 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:38.043789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 797/1607] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:37 2022] PUT /identity/v3/projects/4a9bc1ff5f234d1db581da768a84b07e/users/651bfff9429a4578963c9d6699962a1a/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:38.045378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd851e58-045b-4c61-9bce-c95dc681beaa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.045728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd851e58-045b-4c61-9bce-c95dc681beaa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.046061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd851e58-045b-4c61-9bce-c95dc681beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.103664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fd851e58-045b-4c61-9bce-c95dc681beaa None None] MFA Rules not processed for user `c3c4dc5ead6445ce91b8625a9e170f36`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:38.105202 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fd851e58-045b-4c61-9bce-c95dc681beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:38.203871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:38.205142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3adc-7a15-4f84-bfa3-ee625eeb4ef1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:38.256678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ece8f-f866-4ecc-b799-dd4bb54e7c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:38.257392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ece8f-f866-4ecc-b799-dd4bb54e7c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.257392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ece8f-f866-4ecc-b799-dd4bb54e7c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.257392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ece8f-f866-4ecc-b799-dd4bb54e7c19 admin admin] PATH_INFO: `/v3/projects/3a0c0004225f4bedae9dd5856edb4118/users/d4ca5ffca22746578e67ca4a5d4780f5/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.258211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 798/1608] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:38.260530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede92b5d-ee73-4c9a-b144-460dbcd139c5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.262121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede92b5d-ee73-4c9a-b144-460dbcd139c5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.262655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede92b5d-ee73-4c9a-b144-460dbcd139c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.278915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ece8f-f866-4ecc-b799-dd4bb54e7c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0c0004225f4bedae9dd5856edb4118, user_id=d4ca5ffca22746578e67ca4a5d4780f5, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:38.282609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ece8f-f866-4ecc-b799-dd4bb54e7c19 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:38.285506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ede92b5d-ee73-4c9a-b144-460dbcd139c5 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:38.287269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ede92b5d-ee73-4c9a-b144-460dbcd139c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:38.299927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 799/1609] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:38.306236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede92b5d-ee73-4c9a-b144-460dbcd139c5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:38.320143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 811/1610] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:37 2022] PUT /identity/v3/projects/3a0c0004225f4bedae9dd5856edb4118/users/d4ca5ffca22746578e67ca4a5d4780f5/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:38.332228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f3d3f-c4ad-4793-b3df-8e48ec6d1948 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.333695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f3d3f-c4ad-4793-b3df-8e48ec6d1948 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.333985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f3d3f-c4ad-4793-b3df-8e48ec6d1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.398364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a04f3d3f-c4ad-4793-b3df-8e48ec6d1948 None None] MFA Rules not processed for user `5ba7e99def8f4744946e7b1697f55a44`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:38.399506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a04f3d3f-c4ad-4793-b3df-8e48ec6d1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:38.405955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede92b5d-ee73-4c9a-b144-460dbcd139c5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:38.475370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:38.476203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.476707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.477017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.511657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 812/1611] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:38.514039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750f8b6-f57c-46e2-a0b1-db62d1135e6d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.514696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750f8b6-f57c-46e2-a0b1-db62d1135e6d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.515738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750f8b6-f57c-46e2-a0b1-db62d1135e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.592708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-3750f8b6-f57c-46e2-a0b1-db62d1135e6d None None] MFA Rules not processed for user `651bfff9429a4578963c9d6699962a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:38.596583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3750f8b6-f57c-46e2-a0b1-db62d1135e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:38.600411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:38.602013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:38.609102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:38.616004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 800/1612] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:38.617256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:38.639504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e99473-4354-4c7a-9cd6-180d901c5e1c service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:38.644461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3eab1-eea9-4ba9-86ed-5ad710de11ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:38.645373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3eab1-eea9-4ba9-86ed-5ad710de11ed admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.645837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3eab1-eea9-4ba9-86ed-5ad710de11ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.646118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3eab1-eea9-4ba9-86ed-5ad710de11ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.646805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3eab1-eea9-4ba9-86ed-5ad710de11ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:38.652827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3eab1-eea9-4ba9-86ed-5ad710de11ed admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:38.668282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:38.712078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 801/1613] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:38 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:38.713892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b0a1-c051-473c-8356-e5cd663f7732 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.714153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b0a1-c051-473c-8356-e5cd663f7732 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.714413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b0a1-c051-473c-8356-e5cd663f7732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.721017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 813/1614] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:38.722118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750f8b6-f57c-46e2-a0b1-db62d1135e6d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:38.733526 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750f8b6-f57c-46e2-a0b1-db62d1135e6d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:38.783772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cb01b0a1-c051-473c-8356-e5cd663f7732 None None] MFA Rules not processed for user `d4ca5ffca22746578e67ca4a5d4780f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:38.785208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cb01b0a1-c051-473c-8356-e5cd663f7732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:38.879768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:38.882631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:38.883727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:38.884182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:38.922299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 802/1615] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:38.923253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b0a1-c051-473c-8356-e5cd663f7732 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.018250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.022163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.040500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 814/1616] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:39.042130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.058181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417ff2-0f2e-40a2-b25d-3c34d5ecb8ec service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.065080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.066128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.066826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.067524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.069434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.071996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.078057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b0a1-c051-473c-8356-e5cd663f7732 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.088491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.089165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.089415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.089688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.090336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.092314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.113920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 815/1617] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:39 2022] POST /identity/v3/users => generated 291 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:39.113920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.134006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887f31-8175-411e-a884-7f340def568d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.142047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.143295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.143295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.143843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.145018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.149490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.161480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 803/1618] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:38 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:39.162464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.210657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 816/1619] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:39 2022] POST /identity/v3/projects => generated 407 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:39.212632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.241956 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:39.317135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e68147-cc59-44ab-858e-3d21c6799467 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.319151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5985ecf-7991-4665-902a-87e8dba3c368 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.325703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.326578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.327250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.327473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.327957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.328758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.329087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.329357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.329683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.330762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.333484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.334197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.350120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:39.353305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 817/1620] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:39.355439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.372501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8576e-d45d-4030-859a-e2e6fbff9b0c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.380298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.381152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.381569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.382623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.382949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.385353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.403200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 804/1621] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:39 2022] POST /identity/v3/projects => generated 373 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:39.403200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.428876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 818/1622] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:39 2022] POST /identity/v3/users => generated 292 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:39.430431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.453919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f6dfc-373d-4be5-a68a-1a5ba99af25f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.548321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0ca36-6cc2-4546-b052-e0c6f5d88e2e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.561102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.561591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.561958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.562288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.562943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.565726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.580414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.581590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.581590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.582098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] PATH_INFO: `/v3/projects/7be3a0739cb7432cb652dabc10cd2417/users/c0431aa1305246c1ac69e71e9667a681/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.602822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be3a0739cb7432cb652dabc10cd2417, user_id=c0431aa1305246c1ac69e71e9667a681, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.606400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.614940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 805/1623] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:39 2022] POST /identity/v3/users => generated 310 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:39.615716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.631169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97a818-ef63-4c5e-9062-52e5632b3430 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.643357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 819/1624] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:39 2022] PUT /identity/v3/projects/7be3a0739cb7432cb652dabc10cd2417/users/c0431aa1305246c1ac69e71e9667a681/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:39.645063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.720319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.721099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.721434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.721801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.722465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.725700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.772803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 806/1625] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:39 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:39.772803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.798947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acfd07-5da6-4e09-96b0-da12015531e4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.816085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.817091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.817402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.817674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.819494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.822849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.827871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50476bcb-ff63-4596-ba6f-b5f079f2b7fb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.842234 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.846360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.847290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.847594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.848840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.853933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.857462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 807/1626] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:39.860068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.886552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 820/1627] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:39.889159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.891598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67bfe2-c906-4fa8-b1c9-ddc64dfe49c7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.905897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.906539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.906816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.907091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.908017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.912638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.913566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51677d-8b49-4447-8a31-3e9d440c3f99 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.925952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.926583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.928382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.928685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.931608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:39.935276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:39.941475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 808/1628] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:39.958509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:39.971689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417038d-026f-46d2-84bf-9af09b92d2ba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:39.985569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:39.986278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:39.986555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:39.986827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] PATH_INFO: `/v3/projects/8c09a83758bd49dcb5f14cd0ce7f14d3/users/154a868e40434399b436dc3f0fb6d081/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:39.995697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 821/1629] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:39.995697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.013201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:40.023133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c09a83758bd49dcb5f14cd0ce7f14d3, user_id=154a868e40434399b436dc3f0fb6d081, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.025423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326a547-1fd8-41cb-961b-437d586266e7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.026592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.040844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.045891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.045891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.045891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] PATH_INFO: `/v3/projects/5ec56dea8111409f8e51794a74f81c7f/users/abd7d5312bab49d28f84e2991e31f351/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.049962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 809/1630] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:39 2022] PUT /identity/v3/projects/8c09a83758bd49dcb5f14cd0ce7f14d3/users/154a868e40434399b436dc3f0fb6d081/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:40.055351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.089152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec56dea8111409f8e51794a74f81c7f, user_id=abd7d5312bab49d28f84e2991e31f351, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.098740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.135975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:40.139847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 822/1631] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:39 2022] PUT /identity/v3/projects/5ec56dea8111409f8e51794a74f81c7f/users/abd7d5312bab49d28f84e2991e31f351/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:40.139847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.242761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35545381-68fa-4bc5-9db9-7a0ea86ed65e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.251432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4a104-1859-4032-a33c-2091efaefd56 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.258689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.259312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.260204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.260591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] PATH_INFO: `/v3/projects/54da4830cb5c4615bd6bb8486ee1f677/users/e8c8add347924a7a9f845ceb06d527a0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.266594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.267221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.267502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.267776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] PATH_INFO: `/v3/projects/7be3a0739cb7432cb652dabc10cd2417/users/c0431aa1305246c1ac69e71e9667a681/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.283202 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be3a0739cb7432cb652dabc10cd2417, user_id=c0431aa1305246c1ac69e71e9667a681, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.284642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54da4830cb5c4615bd6bb8486ee1f677, user_id=e8c8add347924a7a9f845ceb06d527a0, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.289606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.293174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.319103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 810/1632] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:40 2022] PUT /identity/v3/projects/7be3a0739cb7432cb652dabc10cd2417/users/c0431aa1305246c1ac69e71e9667a681/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:40.319800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 823/1633] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:40 2022] PUT /identity/v3/projects/54da4830cb5c4615bd6bb8486ee1f677/users/e8c8add347924a7a9f845ceb06d527a0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:40.320152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.323069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.445272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef111207-5ea7-4918-80b9-265cab53fe19 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.460382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.462397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.464630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.465861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.467674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.472284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.494381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4df5d-92af-47eb-97e1-6ee536d13cc6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.504226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d3451-a2b6-434f-9160-ececb54ac9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.505161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3451-a2b6-434f-9160-ececb54ac9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.505615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3451-a2b6-434f-9160-ececb54ac9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.506058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3451-a2b6-434f-9160-ececb54ac9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.506604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 824/1634] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:40.509798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.511667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d3451-a2b6-434f-9160-ececb54ac9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.513581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d3451-a2b6-434f-9160-ececb54ac9c4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.536011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c944f5e-e090-4c24-a7e0-1be72ee06a0f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.543743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 811/1635] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:40.548511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfe651-d548-4c0e-84e1-dad6e6dd2e7b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.552068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfe651-d548-4c0e-84e1-dad6e6dd2e7b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.552406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfe651-d548-4c0e-84e1-dad6e6dd2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.555242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.556309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.560186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.560806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.561875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.567796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.601399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 825/1636] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:40.605380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.623216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-abdfe651-d548-4c0e-84e1-dad6e6dd2e7b None None] MFA Rules not processed for user `c0431aa1305246c1ac69e71e9667a681`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:40.624712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-abdfe651-d548-4c0e-84e1-dad6e6dd2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:40.639368 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ad8df-e377-41c4-8ae5-6ac0b5e75d5c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.664004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.667473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.668223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.668634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] PATH_INFO: `/v3/projects/5ec56dea8111409f8e51794a74f81c7f/users/abd7d5312bab49d28f84e2991e31f351/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.674101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec56dea8111409f8e51794a74f81c7f, user_id=abd7d5312bab49d28f84e2991e31f351, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.688189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.723360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 826/1637] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:40 2022] PUT /identity/v3/projects/5ec56dea8111409f8e51794a74f81c7f/users/abd7d5312bab49d28f84e2991e31f351/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:40.724638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.810304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 812/1638] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:40.810304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfe651-d548-4c0e-84e1-dad6e6dd2e7b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:40.837613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfe651-d548-4c0e-84e1-dad6e6dd2e7b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.856075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:40.908884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9d762-51bc-4300-95fe-155ca7cd1096 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:40.915235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645eae73-c5a4-4f32-9030-5215cce3692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:40.916195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eae73-c5a4-4f32-9030-5215cce3692b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.916643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eae73-c5a4-4f32-9030-5215cce3692b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.917009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645eae73-c5a4-4f32-9030-5215cce3692b admin admin] PATH_INFO: `/v3/projects/8c09a83758bd49dcb5f14cd0ce7f14d3/users/154a868e40434399b436dc3f0fb6d081/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:40.919118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645eae73-c5a4-4f32-9030-5215cce3692b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c09a83758bd49dcb5f14cd0ce7f14d3, user_id=154a868e40434399b436dc3f0fb6d081, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:40.923467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645eae73-c5a4-4f32-9030-5215cce3692b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:40.949708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:40.968082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 827/1639] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:40 2022] PUT /identity/v3/projects/8c09a83758bd49dcb5f14cd0ce7f14d3/users/154a868e40434399b436dc3f0fb6d081/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:40.968082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3471a-fe8c-4c30-bf1a-f2aa83d1aa27 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:40.968082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3471a-fe8c-4c30-bf1a-f2aa83d1aa27 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:40.968082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3471a-fe8c-4c30-bf1a-f2aa83d1aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.018640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:41.019255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.019532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.019796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] PATH_INFO: `/v3/projects/54da4830cb5c4615bd6bb8486ee1f677/users/e8c8add347924a7a9f845ceb06d527a0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.033432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=54da4830cb5c4615bd6bb8486ee1f677, user_id=e8c8add347924a7a9f845ceb06d527a0, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:41.034930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ecd3471a-fe8c-4c30-bf1a-f2aa83d1aa27 None None] MFA Rules not processed for user `abd7d5312bab49d28f84e2991e31f351`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:41.036635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:41.036795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ecd3471a-fe8c-4c30-bf1a-f2aa83d1aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:41.063097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 813/1640] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:40 2022] PUT /identity/v3/projects/54da4830cb5c4615bd6bb8486ee1f677/users/e8c8add347924a7a9f845ceb06d527a0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:41.063593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:41.177658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b64921-0ea8-4322-a06e-77cfd7d9b4bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:41.188103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3c568-52ac-4809-b30d-b34679dbf1e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:41.189504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c568-52ac-4809-b30d-b34679dbf1e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.189982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c568-52ac-4809-b30d-b34679dbf1e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.191453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c568-52ac-4809-b30d-b34679dbf1e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.211306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 828/1641] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:41.215722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc1a9-337b-4886-bfc3-c636ac13bc97 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.226032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc1a9-337b-4886-bfc3-c636ac13bc97 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.227607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc1a9-337b-4886-bfc3-c636ac13bc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.290294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e2acc1a9-337b-4886-bfc3-c636ac13bc97 None None] MFA Rules not processed for user `154a868e40434399b436dc3f0fb6d081`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:41.294499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e2acc1a9-337b-4886-bfc3-c636ac13bc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:41.364366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c568-52ac-4809-b30d-b34679dbf1e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:41.366708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c568-52ac-4809-b30d-b34679dbf1e4 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:41.386905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 814/1642] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:41.389430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2b3d-14f1-4008-98e6-b126f97af41b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.390072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2b3d-14f1-4008-98e6-b126f97af41b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.390604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2b3d-14f1-4008-98e6-b126f97af41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.435574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7f7c2b3d-14f1-4008-98e6-b126f97af41b None None] MFA Rules not processed for user `e8c8add347924a7a9f845ceb06d527a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:41.437984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7f7c2b3d-14f1-4008-98e6-b126f97af41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:41.452553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 829/1643] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:41 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:41.452553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acc1a9-337b-4886-bfc3-c636ac13bc97 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:41.468561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acc1a9-337b-4886-bfc3-c636ac13bc97 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:41.480392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:41.481404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.481801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.482183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.558585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 815/1644] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:41.560374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c2b3d-14f1-4008-98e6-b126f97af41b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:41.567955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:41.576846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c2b3d-14f1-4008-98e6-b126f97af41b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:41.582561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:41.583361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.583863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.584289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.595162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:41.599170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:41.601490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:41.616236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 816/1645] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:41 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:41.625556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:41.627010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:41.651061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 830/1646] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:41.741549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:41.755403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d8ccb-b85c-430d-9f59-659cc024bcd3 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:41.774273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:41.775251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.775665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.776089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.786831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:41.798933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365af0eb-6453-44cf-a3f0-0ee5e786915f service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:41.807686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:41.808324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:41.808638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:41.808911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:41.834390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:41.836259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:41.863598 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 831/1647] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:01:41.991015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:41.992707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:42.022796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 817/1648] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:42.369474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:42.382148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed897d-957c-4366-badc-8ef120e95cf1 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:42.388468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:42.388663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:42.388937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:42.389304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:42.498995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:42.501386 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:42.532316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 832/1649] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:42.943901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:42.958857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abcf84-cdaa-491b-9ee0-5ea1f56c73d8 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:42.964013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f31dc-84f1-4d8d-b798-11dfabd21991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:42.964960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f31dc-84f1-4d8d-b798-11dfabd21991 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:42.965325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f31dc-84f1-4d8d-b798-11dfabd21991 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:42.965822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f31dc-84f1-4d8d-b798-11dfabd21991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:43.155844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f31dc-84f1-4d8d-b798-11dfabd21991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:43.158282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f31dc-84f1-4d8d-b798-11dfabd21991 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:43.208757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 818/1650] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:42 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:44.960726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:45.133456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bde34d-a8db-44e2-aaaa-51c6f80bfe37 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:45.152316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:45.152316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:45.152818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:45.153109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] PATH_INFO: `/v3/users/d4ca5ffca22746578e67ca4a5d4780f5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:45.160025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ca5ffca22746578e67ca4a5d4780f5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:45.165887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: WARNING py.warnings [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 18:01:45.165887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: warnings.warn(msg) Sep 13 18:01:45.165887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: Sep 13 18:01:45.171247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:45.375716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ca5ffca22746578e67ca4a5d4780f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:45.396745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ca5ffca22746578e67ca4a5d4780f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:45.397500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ca5ffca22746578e67ca4a5d4780f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:45.409087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:45.418050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ca5ffca22746578e67ca4a5d4780f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:45.465206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ca5ffca22746578e67ca4a5d4780f5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:45.482857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 833/1651] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:01:44 2022] DELETE /identity/v3/users/d4ca5ffca22746578e67ca4a5d4780f5 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:45.488211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2fe40-2989-41b8-aa08-6944985e765d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:45.488803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2fe40-2989-41b8-aa08-6944985e765d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:45.488803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2fe40-2989-41b8-aa08-6944985e765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:45.524961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7ca2fe40-2989-41b8-aa08-6944985e765d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:45.526297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ca2fe40-2989-41b8-aa08-6944985e765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:45.858993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 819/1652] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:46.020237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:46.025601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:46.225180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3be0-36a4-482f-8f0e-245232588bf6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:46.237121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:46.248793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:46.249079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:46.251531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:46.276758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:46.278561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:46.319749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 834/1653] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:47.346159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0a5bf-a22e-4947-a535-25854eb94d39 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:47.347288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0a5bf-a22e-4947-a535-25854eb94d39 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:47.347806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0a5bf-a22e-4947-a535-25854eb94d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:47.415007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7ca0a5bf-a22e-4947-a535-25854eb94d39 None None] MFA Rules not processed for user `60800cf2774e409d9c1b6f4992669991`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:47.416883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ca0a5bf-a22e-4947-a535-25854eb94d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:47.445973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:47.514700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:47.516713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:47.530717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 820/1654] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:47.540617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723fb6-9333-4891-a24f-ae965615c520 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:47.540930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723fb6-9333-4891-a24f-ae965615c520 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:47.541228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e723fb6-9333-4891-a24f-ae965615c520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:47.542646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123b4a3-7963-428a-a06b-fc496f41982d service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:47.561310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:47.562373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:47.562789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:47.563197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] PATH_INFO: `/v3/users/c3c4dc5ead6445ce91b8625a9e170f36` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:47.564644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c4dc5ead6445ce91b8625a9e170f36)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:47.566929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:47.570042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4e723fb6-9333-4891-a24f-ae965615c520 None None] MFA Rules not processed for user `60800cf2774e409d9c1b6f4992669991`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:47.571282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e723fb6-9333-4891-a24f-ae965615c520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:47.581888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 821/1655] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:47.624791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3696d6-e823-428c-8c0e-837c2933f4a8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:47.625387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3696d6-e823-428c-8c0e-837c2933f4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:47.625673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3696d6-e823-428c-8c0e-837c2933f4a8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:47.628348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 822/1656] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 13 18:01:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:01:47.640570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a50708-bb41-4e6f-ae20-807592b740b3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:47.640910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a50708-bb41-4e6f-ae20-807592b740b3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:47.641187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a50708-bb41-4e6f-ae20-807592b740b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:47.685691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c4dc5ead6445ce91b8625a9e170f36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:47.700718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c4dc5ead6445ce91b8625a9e170f36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:47.701442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c4dc5ead6445ce91b8625a9e170f36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:47.719077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c4dc5ead6445ce91b8625a9e170f36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:47.744457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e5a50708-bb41-4e6f-ae20-807592b740b3 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:47.747619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e5a50708-bb41-4e6f-ae20-807592b740b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:47.753156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f43bc1ae-a8dd-4125-b7d9-7ce81d55d5e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c4dc5ead6445ce91b8625a9e170f36'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:47.766957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 835/1657] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:01:47 2022] DELETE /identity/v3/users/c3c4dc5ead6445ce91b8625a9e170f36 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:01:47.769061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:47.769353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:47.769632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:47.795861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:47.797160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:47.896576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:47.945238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 823/1658] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 13 18:01:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:01:47.956252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a50708-bb41-4e6f-ae20-807592b740b3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:47.959571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 836/1659] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:47.976448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a50708-bb41-4e6f-ae20-807592b740b3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:47.991547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:47.993649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:48.127283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c2e17-0156-4514-9a7a-d3f81ad748cd None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:48.132127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5a576-74b2-4d06-bd88-ddfbe4096e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:48.132576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5a576-74b2-4d06-bd88-ddfbe4096e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:48.132860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5a576-74b2-4d06-bd88-ddfbe4096e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:48.133134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5a576-74b2-4d06-bd88-ddfbe4096e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:48.167613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5a576-74b2-4d06-bd88-ddfbe4096e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:48.170596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5a576-74b2-4d06-bd88-ddfbe4096e0b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:48.188092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bd6c0-c7ad-4e57-a5a0-9d875d502a9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:48.188795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bd6c0-c7ad-4e57-a5a0-9d875d502a9b service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:48.189071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bd6c0-c7ad-4e57-a5a0-9d875d502a9b service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:48.189334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bd6c0-c7ad-4e57-a5a0-9d875d502a9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:48.200838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 837/1660] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:48.201051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ab4bd6c0-c7ad-4e57-a5a0-9d875d502a9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:48.263671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:48.306259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bd6c0-c7ad-4e57-a5a0-9d875d502a9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:48.307928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bd6c0-c7ad-4e57-a5a0-9d875d502a9b service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:48.320331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 824/1661] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:01:48.686113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184396e1-b686-4396-8dcf-221038b5375b None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:48.686669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184396e1-b686-4396-8dcf-221038b5375b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:48.686743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184396e1-b686-4396-8dcf-221038b5375b None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:48.688259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 838/1662] 10.222.0.43 () {54 vars in 912 bytes} [Tue Sep 13 18:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:01:48.694598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53bac-55e9-4df0-a4f2-52fdd1bb3c68 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:48.695087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53bac-55e9-4df0-a4f2-52fdd1bb3c68 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:48.695160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53bac-55e9-4df0-a4f2-52fdd1bb3c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:48.727671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-82a53bac-55e9-4df0-a4f2-52fdd1bb3c68 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:48.732308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-82a53bac-55e9-4df0-a4f2-52fdd1bb3c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:48.766527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 825/1663] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:48.769623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184396e1-b686-4396-8dcf-221038b5375b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:48.773867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-184396e1-b686-4396-8dcf-221038b5375b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:48.790438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:48.790438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:48.791367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:48.861764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] MFA Rules not processed for user `e68ecd9b3ca643539b109f75bb242d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:48.863719 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:48.978850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184396e1-b686-4396-8dcf-221038b5375b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:48.986207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1918d-0835-463c-9fb3-9c7f4cb95ccd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:48.989656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1918d-0835-463c-9fb3-9c7f4cb95ccd service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:48.990145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1918d-0835-463c-9fb3-9c7f4cb95ccd service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:48.990550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1918d-0835-463c-9fb3-9c7f4cb95ccd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.001763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 826/1664] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:49.001763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:49.011251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:49.033329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1918d-0835-463c-9fb3-9c7f4cb95ccd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:49.036093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1918d-0835-463c-9fb3-9c7f4cb95ccd service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:49.053832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 839/1665] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:49.056723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980b8de-5f11-4e48-b57d-e6346a1863c1 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:49.057254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980b8de-5f11-4e48-b57d-e6346a1863c1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:49.061140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980b8de-5f11-4e48-b57d-e6346a1863c1 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.062809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 840/1666] 10.222.0.43 () {54 vars in 962 bytes} [Tue Sep 13 18:01:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:01:49.065824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2201fa-b319-49dd-88d1-7adfa85b5a4a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:49.074148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686083-df34-40b7-a53a-5c0e3f20afc3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:49.077734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686083-df34-40b7-a53a-5c0e3f20afc3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:49.078035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686083-df34-40b7-a53a-5c0e3f20afc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.079508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:49.080185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:49.080491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:49.080744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] PATH_INFO: `/v3/projects/3a0c0004225f4bedae9dd5856edb4118` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.086851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0c0004225f4bedae9dd5856edb4118)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:49.088006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 18:01:49.088006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Sep 13 18:01:49.088006 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: Sep 13 18:01:49.094191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:49.156327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d3686083-df34-40b7-a53a-5c0e3f20afc3 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:49.161629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d3686083-df34-40b7-a53a-5c0e3f20afc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:49.219880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 841/1667] 10.222.0.43 () {58 vars in 1059 bytes} [Tue Sep 13 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:01:49.229085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3686083-df34-40b7-a53a-5c0e3f20afc3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:49.258544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3686083-df34-40b7-a53a-5c0e3f20afc3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:49.275340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0c0004225f4bedae9dd5856edb4118', 'request_id': 'req-71b90be4-0a43-400d-9521-5f50928716df'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.301243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0c0004225f4bedae9dd5856edb4118', 'request_id': 'req-71b90be4-0a43-400d-9521-5f50928716df'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.334703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0c0004225f4bedae9dd5856edb4118', 'request_id': 'req-71b90be4-0a43-400d-9521-5f50928716df'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.361881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] The token cache is being invalidate because project 3a0c0004225f4bedae9dd5856edb4118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:01:49.362321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.365086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.369404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 827/1668] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:01:48 2022] DELETE /identity/v3/projects/3a0c0004225f4bedae9dd5856edb4118 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:49.373624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:49.377384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:49.425428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2cf56-9677-4c42-9f27-cc36c47ede6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:49.426991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2cf56-9677-4c42-9f27-cc36c47ede6f service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:49.429282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2cf56-9677-4c42-9f27-cc36c47ede6f service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:49.429282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2cf56-9677-4c42-9f27-cc36c47ede6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.435147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d8f2cf56-9677-4c42-9f27-cc36c47ede6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:49.457948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:49.483291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:49.535084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b90be4-0a43-400d-9521-5f50928716df admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:49.547250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:49.548937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:49.550243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:49.550657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] PATH_INFO: `/v3/users/e8c8add347924a7a9f845ceb06d527a0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.552854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c8add347924a7a9f845ceb06d527a0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:49.557059 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:49.597038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2cf56-9677-4c42-9f27-cc36c47ede6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:49.598684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2cf56-9677-4c42-9f27-cc36c47ede6f service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:49.620877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 842/1669] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:49.677101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c8add347924a7a9f845ceb06d527a0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.686816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56111-da97-4638-bd8d-46ff13f8b49a None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:49.687287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56111-da97-4638-bd8d-46ff13f8b49a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:49.687360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56111-da97-4638-bd8d-46ff13f8b49a None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.694371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 843/1670] 10.222.0.43 () {54 vars in 912 bytes} [Tue Sep 13 18:01:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:01:49.695384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c8add347924a7a9f845ceb06d527a0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.696103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c8add347924a7a9f845ceb06d527a0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.701667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:49.701964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:49.702206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:49.707666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c8add347924a7a9f845ceb06d527a0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.750967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c8add347924a7a9f845ceb06d527a0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:49.773967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] MFA Rules not processed for user `a4b10367002d4f64a82b3e9eca00ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:49.775297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:49.777605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 828/1671] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:01:49 2022] DELETE /identity/v3/users/e8c8add347924a7a9f845ceb06d527a0 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:49.983509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 844/1672] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:01:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:01:50.025559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:50.027075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:50.052855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f40adc-6d8f-4c2c-8fb7-253969427f72 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:50.070387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:50.071134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:50.075922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:50.076252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:50.102414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:50.106817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:50.131498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 829/1673] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:50.898257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:50.899950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:51.095549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b1d7-b0de-46dc-a2dd-1d31b28b5694 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:51.113965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:51.114204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:51.114477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:51.114757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] PATH_INFO: `/v3/projects/15fa8103e3a94cb8ac8fa73a29a1877d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:51.116091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] RBAC: Authorizing `identity:delete_project(project_id=15fa8103e3a94cb8ac8fa73a29a1877d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:51.118236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:51.252336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:51.256001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15fa8103e3a94cb8ac8fa73a29a1877d', 'request_id': 'req-c36099b4-d3ff-4dfe-b295-37af8b18d11f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.298755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15fa8103e3a94cb8ac8fa73a29a1877d', 'request_id': 'req-c36099b4-d3ff-4dfe-b295-37af8b18d11f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.316222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15fa8103e3a94cb8ac8fa73a29a1877d', 'request_id': 'req-c36099b4-d3ff-4dfe-b295-37af8b18d11f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.341220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] The token cache is being invalidate because project 15fa8103e3a94cb8ac8fa73a29a1877d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:01:51.341496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.343330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.348947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 845/1674] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:01:50 2022] DELETE /identity/v3/projects/15fa8103e3a94cb8ac8fa73a29a1877d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:01:51.349872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:51.373177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c05ad-9a54-4128-8305-8147f1d438cd service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:51.445152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:51.446563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:51.567331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36099b4-d3ff-4dfe-b295-37af8b18d11f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:51.575417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:51.576161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:51.576410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:51.579235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] PATH_INFO: `/v3/projects/54da4830cb5c4615bd6bb8486ee1f677` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:51.579305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:51.579899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:51.580065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:51.580329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] PATH_INFO: `/v3/users/c0431aa1305246c1ac69e71e9667a681` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:51.580775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=54da4830cb5c4615bd6bb8486ee1f677)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:51.585561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0431aa1305246c1ac69e71e9667a681)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:51.587679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:51.588976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:51.664134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54da4830cb5c4615bd6bb8486ee1f677', 'request_id': 'req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.693268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54da4830cb5c4615bd6bb8486ee1f677', 'request_id': 'req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.699412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:51.701844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0431aa1305246c1ac69e71e9667a681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.708507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54da4830cb5c4615bd6bb8486ee1f677', 'request_id': 'req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.715481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0431aa1305246c1ac69e71e9667a681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.716167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0431aa1305246c1ac69e71e9667a681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.736630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0431aa1305246c1ac69e71e9667a681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.757026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] The token cache is being invalidate because project 54da4830cb5c4615bd6bb8486ee1f677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:01:51.757339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.758979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ae8a7ed2-6f1e-4efd-b0d6-314306e57b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.759948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0431aa1305246c1ac69e71e9667a681'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:51.762291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 846/1675] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:01:51 2022] DELETE /identity/v3/projects/54da4830cb5c4615bd6bb8486ee1f677 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:51.763567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd1d21-fd8b-47eb-b117-f21ca5e52906 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:51.763850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd1d21-fd8b-47eb-b117-f21ca5e52906 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:51.764124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd1d21-fd8b-47eb-b117-f21ca5e52906 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:51.769406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 830/1676] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:01:51 2022] DELETE /identity/v3/users/c0431aa1305246c1ac69e71e9667a681 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:51.770311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:51.772783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 847/1677] 10.222.0.43 () {54 vars in 912 bytes} [Tue Sep 13 18:01:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:01:51.774174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac738e-1814-4556-af3b-5766ffda0dd2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:51.777061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:51.779618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac738e-1814-4556-af3b-5766ffda0dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:51.780044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac738e-1814-4556-af3b-5766ffda0dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:51.826495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0dac738e-1814-4556-af3b-5766ffda0dd2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:51.827755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0dac738e-1814-4556-af3b-5766ffda0dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:51.938501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af8c69-60b6-436e-9c99-e27dc7279cd6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:51.948023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2c173-d178-4966-a717-e93b6ff34771 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:51.948966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c173-d178-4966-a717-e93b6ff34771 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:51.949339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c173-d178-4966-a717-e93b6ff34771 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:51.952509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2c173-d178-4966-a717-e93b6ff34771 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:51.955536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-edc2c173-d178-4966-a717-e93b6ff34771 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:51.999688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c173-d178-4966-a717-e93b6ff34771 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:52.005352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2c173-d178-4966-a717-e93b6ff34771 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:52.031207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 848/1678] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:52.031207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1314d0-7eee-4a15-9705-1fa56d08f959 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:52.031207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1314d0-7eee-4a15-9705-1fa56d08f959 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:52.031207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1314d0-7eee-4a15-9705-1fa56d08f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:52.068469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:52.088762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 831/1679] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:52.094808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-be1314d0-7eee-4a15-9705-1fa56d08f959 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:52.096048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be1314d0-7eee-4a15-9705-1fa56d08f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:52.101100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77dfd-d2f2-40f6-96f4-11a4a91fc67e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:52.106118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77dfd-d2f2-40f6-96f4-11a4a91fc67e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:52.106647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c77dfd-d2f2-40f6-96f4-11a4a91fc67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:52.182479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-90c77dfd-d2f2-40f6-96f4-11a4a91fc67e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:52.186424 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90c77dfd-d2f2-40f6-96f4-11a4a91fc67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:52.241695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 832/1680] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:52.242892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77dfd-d2f2-40f6-96f4-11a4a91fc67e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:52.279397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 849/1681] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:01:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:52.282184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c77dfd-d2f2-40f6-96f4-11a4a91fc67e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:52.302197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:52.302843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:52.309337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:52.309743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:52.313880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:52.314583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:52.318525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:52.356558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1314d0-7eee-4a15-9705-1fa56d08f959 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:52.363730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 833/1682] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:52.374053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1314d0-7eee-4a15-9705-1fa56d08f959 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:52.387341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:52.387798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:52.388075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:52.388341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:52.394774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:52.401606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:52.403774 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:52.411229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:52.504945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 850/1683] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:52 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:52.510704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:52.682304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4001fe5-aeb7-4b08-8e88-3814ee3aa894 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:52.741359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ecc24-688c-4173-bb71-aa677ee1088d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:52.774456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:52.775164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:52.778485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:52.779306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:52.784195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:52.794651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:52.895922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 851/1684] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:52 2022] POST /identity/v3/users => generated 283 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:52.897178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:52.926144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:52.950407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23960-caa3-49ee-a390-37e83e5f094d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:52.952928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81e71c-7332-4748-a324-f189cb008d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:52.957184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81e71c-7332-4748-a324-f189cb008d00 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:52.958007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81e71c-7332-4748-a324-f189cb008d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:52.958289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81e71c-7332-4748-a324-f189cb008d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:52.971565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:52.972142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:52.972436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:52.972709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:52.973365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:52.980085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:53.053350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 852/1685] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:53.053350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:53.073829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5e8d5-ff3a-4564-bcdc-b32eac4ddb68 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:53.086672 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:53.087461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:53.089557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:53.089557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:53.094174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:53.102101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:53.130725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 853/1686] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:53.137438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:53.160982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602ed6c-1aca-4376-a0e6-107d3ea411b7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:53.177352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7472485-2a94-4e4c-af09-41977b73d947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:53.178719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7472485-2a94-4e4c-af09-41977b73d947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:53.180161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7472485-2a94-4e4c-af09-41977b73d947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:53.180603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7472485-2a94-4e4c-af09-41977b73d947 admin admin] PATH_INFO: `/v3/projects/56bd882fb4e44545b630d8d880f0d030/users/ddc6675e8071417297fedb5b7ed6eaf0/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:53.188632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81e71c-7332-4748-a324-f189cb008d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:53.191803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81e71c-7332-4748-a324-f189cb008d00 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:53.247446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7472485-2a94-4e4c-af09-41977b73d947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bd882fb4e44545b630d8d880f0d030, user_id=ddc6675e8071417297fedb5b7ed6eaf0, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:53.250639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 834/1687] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 850 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:01:53.253772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7472485-2a94-4e4c-af09-41977b73d947 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:53.315767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f706-285c-4489-918d-725e0c559d85 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:53.316098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f706-285c-4489-918d-725e0c559d85 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:53.316337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f706-285c-4489-918d-725e0c559d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:53.319303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 854/1688] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:53 2022] PUT /identity/v3/projects/56bd882fb4e44545b630d8d880f0d030/users/ddc6675e8071417297fedb5b7ed6eaf0/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:53.371620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6869f706-285c-4489-918d-725e0c559d85 None None] MFA Rules not processed for user `ddc6675e8071417297fedb5b7ed6eaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:53.373873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6869f706-285c-4489-918d-725e0c559d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:53.394902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d9db6-d203-425e-bb92-4eda29dfffe5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:53.395185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d9db6-d203-425e-bb92-4eda29dfffe5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:53.397470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d9db6-d203-425e-bb92-4eda29dfffe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:53.465130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8d9d9db6-d203-425e-bb92-4eda29dfffe5 None None] MFA Rules not processed for user `0625b3113ce044bbac4f6142eaade11c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:53.467022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d9d9db6-d203-425e-bb92-4eda29dfffe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:53.479672 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:53.633722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 855/1689] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:53.640798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045a6c-39e3-4046-a1ec-a8d8831327da None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:53.647207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045a6c-39e3-4046-a1ec-a8d8831327da None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:53.647836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045a6c-39e3-4046-a1ec-a8d8831327da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:53.666303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 835/1690] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:53.667396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869f706-285c-4489-918d-725e0c559d85 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:53.669400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6869f706-285c-4489-918d-725e0c559d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:53.676187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6a045a6c-39e3-4046-a1ec-a8d8831327da None None] MFA Rules not processed for user `60800cf2774e409d9c1b6f4992669991`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:53.680594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6a045a6c-39e3-4046-a1ec-a8d8831327da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:53.789515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 856/1691] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:53.791076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045a6c-39e3-4046-a1ec-a8d8831327da None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:53.890281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045a6c-39e3-4046-a1ec-a8d8831327da None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:53.903305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:53.903999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:53.904280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:53.904602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:53.905423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:53.907822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:53.934463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869f706-285c-4489-918d-725e0c559d85 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:53.958913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:53.959616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:53.959861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:53.960114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:53.979895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:53.987583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:53.990456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 857/1692] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:53 2022] POST /identity/v3/projects => generated 363 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:53.991938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:53.997343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:54.026226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 836/1693] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:54.028152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:54.052172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:54.077893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b2ab1-2136-435b-8661-df63c1702173 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:54.142541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:54.227580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc2940-e592-41be-a978-9df1693625a0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:54.242395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:54.243324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.243655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.243996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] PATH_INFO: `/v3/projects/7be3a0739cb7432cb652dabc10cd2417` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.247524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be3a0739cb7432cb652dabc10cd2417)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:54.267254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:54.285721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:54.286481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.287134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.287451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.288131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:54.296754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:54.353487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 837/1694] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:54 2022] POST /identity/v3/users => generated 288 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:54.354946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:54.399063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8533f7-0a5b-4935-9f9b-c8cc7655afef admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:54.402397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be3a0739cb7432cb652dabc10cd2417', 'request_id': 'req-62866517-735a-41a9-ba80-49041054c0cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:54.429223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be3a0739cb7432cb652dabc10cd2417', 'request_id': 'req-62866517-735a-41a9-ba80-49041054c0cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:54.457194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be3a0739cb7432cb652dabc10cd2417', 'request_id': 'req-62866517-735a-41a9-ba80-49041054c0cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:54.504067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] The token cache is being invalidate because project 7be3a0739cb7432cb652dabc10cd2417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:01:54.504373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:54.510539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-62866517-735a-41a9-ba80-49041054c0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:54.521013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 858/1695] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:01:53 2022] DELETE /identity/v3/projects/7be3a0739cb7432cb652dabc10cd2417 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:54.530814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed338ac6-1f86-4700-8139-f8ec2850f00f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.532026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed338ac6-1f86-4700-8139-f8ec2850f00f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.532316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed338ac6-1f86-4700-8139-f8ec2850f00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.579527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ed338ac6-1f86-4700-8139-f8ec2850f00f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:54.581321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ed338ac6-1f86-4700-8139-f8ec2850f00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:54.602102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:54.602744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.603338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.603338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.603940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:54.605620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:54.644732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 838/1696] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:54.654119 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:54.660859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:54.694151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67d32c-1110-4314-9884-9f51bb618540 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:54.706813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:54.707511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.707796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.708131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] PATH_INFO: `/v3/projects/5f3d7c67967844b48550a0b221283464/users/6908e0ecf87043cc8c6171c37e702b58/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.744515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3d7c67967844b48550a0b221283464, user_id=6908e0ecf87043cc8c6171c37e702b58, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:54.747701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:54.782918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 859/1697] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:54.792722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042f20c-bd57-4408-af43-8cc60e15314c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.793059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042f20c-bd57-4408-af43-8cc60e15314c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.793363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042f20c-bd57-4408-af43-8cc60e15314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.810642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 839/1698] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:54 2022] PUT /identity/v3/projects/5f3d7c67967844b48550a0b221283464/users/6908e0ecf87043cc8c6171c37e702b58/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:54.815347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:54.825664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7042f20c-bd57-4408-af43-8cc60e15314c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:54.827648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7042f20c-bd57-4408-af43-8cc60e15314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:54.960101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 860/1699] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:54.964059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042f20c-bd57-4408-af43-8cc60e15314c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:54.972385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a65b0-7574-42bf-ac63-ad8b7925f428 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:54.981703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042f20c-bd57-4408-af43-8cc60e15314c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:54.985027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaa317-e18a-40e2-83ab-f73a5548d253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:54.985653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa317-e18a-40e2-83ab-f73a5548d253 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.985934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa317-e18a-40e2-83ab-f73a5548d253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.986209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaa317-e18a-40e2-83ab-f73a5548d253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.986886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaa317-e18a-40e2-83ab-f73a5548d253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:54.988549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaa317-e18a-40e2-83ab-f73a5548d253 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:54.989328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:54.990385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:54.990805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:54.991830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:54.993264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:01:54.994237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:54.996475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:55.006830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:55.013973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 840/1700] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:55.015506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b1a8d-b0f0-41cb-baab-238230b0a92c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.015893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b1a8d-b0f0-41cb-baab-238230b0a92c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.016160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b1a8d-b0f0-41cb-baab-238230b0a92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.021927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 861/1701] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:01:55.023150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:55.025114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:55.045143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-545b1a8d-b0f0-41cb-baab-238230b0a92c None None] MFA Rules not processed for user `a4b10367002d4f64a82b3e9eca00ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:55.054019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6a7ea-d7c6-49a2-aeac-4cff69d14951 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:55.061495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-545b1a8d-b0f0-41cb-baab-238230b0a92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:55.064989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:55.065675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.065951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.066220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] PATH_INFO: `/v3/projects/5f3d7c67967844b48550a0b221283464/users/6908e0ecf87043cc8c6171c37e702b58/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.069267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3d7c67967844b48550a0b221283464, user_id=6908e0ecf87043cc8c6171c37e702b58, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:55.077723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:55.107461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 862/1702] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:55 2022] PUT /identity/v3/projects/5f3d7c67967844b48550a0b221283464/users/6908e0ecf87043cc8c6171c37e702b58/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:55.110460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:55.227554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 841/1703] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:01:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:55.230439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.230936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.231059 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.246084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492b020-3061-4ba4-a8c8-0bd0df66d35b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:55.263056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:55.264019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.264370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.264733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.265606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:55.268403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:55.311707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] MFA Rules not processed for user `6908e0ecf87043cc8c6171c37e702b58`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:55.313504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:55.327182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 863/1704] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:55 2022] POST /identity/v3/projects => generated 379 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:55.331968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:55.441250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 842/1705] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:55.455640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:55.460358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:55.525116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49745f01-1962-4631-8434-da3c0a2b7018 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:55.531999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:55.532773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.533119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.533474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.534241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:55.536911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:55.592969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 864/1706] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:55 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:55.596270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:55.600749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:55.625555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8770277-b742-493e-b9f8-88e8ff6e97e0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:55.758389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86950d60-3203-49a5-9909-bf7af77f310c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:55.760755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86950d60-3203-49a5-9909-bf7af77f310c service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.760755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86950d60-3203-49a5-9909-bf7af77f310c service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.760755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86950d60-3203-49a5-9909-bf7af77f310c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.778748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:55.832550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cc4b4-3053-4ddd-93d3-cdc84cd22400 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:55.845223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:55.845881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.846165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.846445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.883600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:55.887480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:55.919927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 865/1707] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:55.925004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:55.942154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:55.964808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9a1b4-ef88-4810-a72a-50611dc47de7 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:55.967834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86950d60-3203-49a5-9909-bf7af77f310c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:55.976083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:55.978285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86950d60-3203-49a5-9909-bf7af77f310c service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:55.979155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:55.980079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:55.983496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:55.984664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:55.986760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:56.009737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 843/1708] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:56.015563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 866/1709] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:56.020703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:56.022920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902610-bfc0-4b81-8622-482e259031dd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:56.023374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902610-bfc0-4b81-8622-482e259031dd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:56.023542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b902610-bfc0-4b81-8622-482e259031dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:56.043838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e4747-7a83-497d-8848-49614d7008d2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:56.055856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8b902610-bfc0-4b81-8622-482e259031dd None None] MFA Rules not processed for user `e68ecd9b3ca643539b109f75bb242d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:56.058414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8b902610-bfc0-4b81-8622-482e259031dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:56.070049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:56.074201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:56.074756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:56.075408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:56.076292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:56.080133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:56.133829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 867/1710] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:56.134951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:56.158232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3ad17-efed-4d8e-9dd1-0c3631523759 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:56.171608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0535cd9-a22a-4883-855b-8cd79685a86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:56.172485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535cd9-a22a-4883-855b-8cd79685a86e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:56.172942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535cd9-a22a-4883-855b-8cd79685a86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:56.173313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535cd9-a22a-4883-855b-8cd79685a86e admin admin] PATH_INFO: `/v3/projects/3424d6aa3dc34ab8bf8b255ac906736d/users/274ed1b7000448e8bce85786d359f281/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:56.188413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 844/1711] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:56.247119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0535cd9-a22a-4883-855b-8cd79685a86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3424d6aa3dc34ab8bf8b255ac906736d, user_id=274ed1b7000448e8bce85786d359f281, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:56.259454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0535cd9-a22a-4883-855b-8cd79685a86e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:56.284382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:56.285112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:56.285480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:56.311857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 868/1712] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:56 2022] PUT /identity/v3/projects/3424d6aa3dc34ab8bf8b255ac906736d/users/274ed1b7000448e8bce85786d359f281/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:56.323220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5888d-4e40-49a8-ac87-104264dbef57 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:56.323946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5888d-4e40-49a8-ac87-104264dbef57 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:56.325393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5888d-4e40-49a8-ac87-104264dbef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:56.407448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a2d5888d-4e40-49a8-ac87-104264dbef57 None None] MFA Rules not processed for user `274ed1b7000448e8bce85786d359f281`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:56.407964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a2d5888d-4e40-49a8-ac87-104264dbef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:56.433661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:56.437214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:56.633364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 869/1713] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:56.637423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5888d-4e40-49a8-ac87-104264dbef57 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:56.717910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 845/1714] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:01:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:56.721082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:56.725251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:56.818790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:56.951708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ba541-4ef0-4009-8523-ca4b3ec7fb9b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:56.973704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:56.974371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:56.974646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:56.974917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] PATH_INFO: `/v3/users/651bfff9429a4578963c9d6699962a1a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:56.975539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5888d-4e40-49a8-ac87-104264dbef57 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:56.976104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=651bfff9429a4578963c9d6699962a1a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:56.978113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:57.003941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:57.004716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:57.005281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:57.005419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:57.006693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:57.012351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:57.131534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 870/1715] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:01:56 2022] POST /identity/v3/projects => generated 379 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:01:57.137163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:57.145765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:57.147649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651bfff9429a4578963c9d6699962a1a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:57.163032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651bfff9429a4578963c9d6699962a1a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:57.164781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651bfff9429a4578963c9d6699962a1a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:57.199836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651bfff9429a4578963c9d6699962a1a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:57.228969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651bfff9429a4578963c9d6699962a1a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:57.247784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 846/1716] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:01:56 2022] DELETE /identity/v3/users/651bfff9429a4578963c9d6699962a1a => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:57.248879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:57.253693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:57.266839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:57.460965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306add-777f-4f63-8aa8-194b2eb0d438 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:57.474510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcfb85-1cfe-4534-81a0-7e00c99a8668 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:57.490665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c685ff-aefb-49e0-b451-125aa79ac9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:57.491323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c685ff-aefb-49e0-b451-125aa79ac9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:57.491601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c685ff-aefb-49e0-b451-125aa79ac9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:57.491879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c685ff-aefb-49e0-b451-125aa79ac9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:57.492787 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c685ff-aefb-49e0-b451-125aa79ac9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:57.494856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c685ff-aefb-49e0-b451-125aa79ac9f6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:57.543432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 847/1717] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:01:57 2022] POST /identity/v3/users => generated 296 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:01:57.545473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72109-bbeb-4bf8-bb2f-9a2452bb62ab None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:57.545928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72109-bbeb-4bf8-bb2f-9a2452bb62ab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:57.546566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e72109-bbeb-4bf8-bb2f-9a2452bb62ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:57.589815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-06e72109-bbeb-4bf8-bb2f-9a2452bb62ab None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:57.595450 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-06e72109-bbeb-4bf8-bb2f-9a2452bb62ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:57.614599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 848/1718] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:57.617124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb56263-d52f-4a5d-922c-ac75c50cea6d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:57.619283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb56263-d52f-4a5d-922c-ac75c50cea6d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:57.619700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb56263-d52f-4a5d-922c-ac75c50cea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:57.651367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:57.652141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:57.652914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:57.653175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:57.658451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:57.705416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1fb56263-d52f-4a5d-922c-ac75c50cea6d None None] MFA Rules not processed for user `51db5ef052144879b53979268ac4ceb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:57.706659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1fb56263-d52f-4a5d-922c-ac75c50cea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:57.750842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:57.855261 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:57.857171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:57.891719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 871/1719] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:01:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:01:57.893337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:57.895139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 849/1720] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:57.896857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb56263-d52f-4a5d-922c-ac75c50cea6d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:57.909796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:57.922974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb56263-d52f-4a5d-922c-ac75c50cea6d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:57.936307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:57.939246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:57.939582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:57.939858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:57.944495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:57.946323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:57.985842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 850/1721] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:57.991684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:57.993825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b61abd-14d4-40aa-8ffd-6c92ba625601 service swift] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:57.996052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:58.019591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:58.020626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:58.021864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:58.026053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] PATH_INFO: `/v3/users/ddc6675e8071417297fedb5b7ed6eaf0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:58.037013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc6675e8071417297fedb5b7ed6eaf0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:58.047917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:58.216222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc6675e8071417297fedb5b7ed6eaf0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:58.228110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:58.240396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc6675e8071417297fedb5b7ed6eaf0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:58.241664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc6675e8071417297fedb5b7ed6eaf0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:58.261987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e92730-68f0-408f-a20c-6639acae88bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:58.273868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:58.277110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc6675e8071417297fedb5b7ed6eaf0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:58.277829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:58.278916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:58.280212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:58.285073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:58.326699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:58.327502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4771278a-50fb-4f4d-b3a2-78521314c15b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc6675e8071417297fedb5b7ed6eaf0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:01:58.332502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:58.349844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 872/1722] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:01:57 2022] DELETE /identity/v3/users/ddc6675e8071417297fedb5b7ed6eaf0 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:58.352093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7130-a8d2-403c-9ede-fad09c4c3dfa None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:58.352661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7130-a8d2-403c-9ede-fad09c4c3dfa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:58.353426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae7130-a8d2-403c-9ede-fad09c4c3dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:58.382651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 851/1723] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:58.386625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:58.391906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:58.421488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-eeae7130-a8d2-403c-9ede-fad09c4c3dfa None None] MFA Rules not processed for user `51db5ef052144879b53979268ac4ceb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:58.428884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eeae7130-a8d2-403c-9ede-fad09c4c3dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:58.432247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cddd5b-1f89-4eed-8500-aeda142b6e93 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:58.633906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 873/1724] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:58.635636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f021a4c-fd91-44f8-b5e5-a5a0dee5eab8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:58.637714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f021a4c-fd91-44f8-b5e5-a5a0dee5eab8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:58.638895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f021a4c-fd91-44f8-b5e5-a5a0dee5eab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:58.685204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:58.685761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:58.692617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:58.692984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] PATH_INFO: `/v3/projects/8d55c3565cb24c9db82d46fc9b8dd404/users/ff13187ee695496f86064c673a5391d7/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:58.693990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4f021a4c-fd91-44f8-b5e5-a5a0dee5eab8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:58.696615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4f021a4c-fd91-44f8-b5e5-a5a0dee5eab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:58.718230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 874/1725] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:58.746946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d55c3565cb24c9db82d46fc9b8dd404, user_id=ff13187ee695496f86064c673a5391d7, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:58.756472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:58.781862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 852/1726] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:58 2022] PUT /identity/v3/projects/8d55c3565cb24c9db82d46fc9b8dd404/users/ff13187ee695496f86064c673a5391d7/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:58.786301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f021a4c-fd91-44f8-b5e5-a5a0dee5eab8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:58.808961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f021a4c-fd91-44f8-b5e5-a5a0dee5eab8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:58.830698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:58.833474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:58.908805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:59.003516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11439822-254f-455a-bed9-6629f3556f27 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:59.012485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91cc2-a72b-4520-9db1-1137b32749c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:59.013118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91cc2-a72b-4520-9db1-1137b32749c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:59.013583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91cc2-a72b-4520-9db1-1137b32749c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:59.013859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91cc2-a72b-4520-9db1-1137b32749c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:59.019867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:59.021054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:59.021581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:59.021999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:59.022859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:59.024764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:59.033881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91cc2-a72b-4520-9db1-1137b32749c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:59.038095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91cc2-a72b-4520-9db1-1137b32749c9 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:59.056202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 875/1727] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:01:59.059873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 853/1728] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:59.060395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:59.092723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a5f90-f0a8-41a1-8765-14c25b2d25ed admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:59.103339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:59.104255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:59.104560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:59.104857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] PATH_INFO: `/v3/projects/8d55c3565cb24c9db82d46fc9b8dd404/users/ff13187ee695496f86064c673a5391d7/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:59.116215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d55c3565cb24c9db82d46fc9b8dd404, user_id=ff13187ee695496f86064c673a5391d7, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:59.119483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:59.148233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 876/1729] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:01:59 2022] PUT /identity/v3/projects/8d55c3565cb24c9db82d46fc9b8dd404/users/ff13187ee695496f86064c673a5391d7/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:01:59.157520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:59.157880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:59.158225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:59.164706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:59.167056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:59.271996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] MFA Rules not processed for user `ff13187ee695496f86064c673a5391d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:01:59.277849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:59.300215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:01:59.427019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11141e-f5fc-4a99-861d-eca0eef6e859 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:59.509850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:59.523326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:59.524556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:59.528728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:59.539265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:59.625326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 854/1730] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:01:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:01:59.641201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:01:59.646049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:01:59.849487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:59.854720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:01:59.902963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c21d8c-10eb-4fb1-9ddb-4b0dbbeaeae5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:01:59.911862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 877/1731] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:01:59.919813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:01:59.920255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:01:59.920561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:01:59.920834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:01:59.970975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:01:59.972534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:00.077979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 855/1732] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:01:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:01.092668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:01.115145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e32b-bbd7-4c61-8b04-89b0c9140ea3 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:01.132904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff13187ee695496f86064c673a5391d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d55c3565cb24c9db82d46fc9b8dd404', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:01.135154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:01.136228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:01.138319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:01.142433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:01.144144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:01.152291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:01.195535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 878/1733] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:01.199008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:01.213087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa639af-35e0-4307-9848-8f3c930b4a4f service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:01.232855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62b09a-331b-47d4-b568-41df54e94275 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff13187ee695496f86064c673a5391d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d55c3565cb24c9db82d46fc9b8dd404', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:01.235397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62b09a-331b-47d4-b568-41df54e94275 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:01.237885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62b09a-331b-47d4-b568-41df54e94275 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:01.238196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62b09a-331b-47d4-b568-41df54e94275 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:01.240969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62b09a-331b-47d4-b568-41df54e94275 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:01.249202 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62b09a-331b-47d4-b568-41df54e94275 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:01.348330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 856/1734] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:01 2022] POST /identity/v3/projects => generated 355 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:01.351114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:01.374921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e8c4f-c947-4220-a72e-ea8e4ed72525 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:01.553690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:01.553904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:01.554219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:01.611500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff13187ee695496f86064c673a5391d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d55c3565cb24c9db82d46fc9b8dd404', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:01.612108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:01.612395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:01.612730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:01.619197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:01.619459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:01.626144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:01.628990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] MFA Rules not processed for user `a4b10367002d4f64a82b3e9eca00ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:01.630536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:01.685038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 879/1735] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:02:01.770840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:01.787099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbd0ed-c70c-4251-a16a-9a054416df32 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:01.788914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:01.821259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff13187ee695496f86064c673a5391d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d55c3565cb24c9db82d46fc9b8dd404', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:01.821931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:01.822264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:01.822582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] PATH_INFO: `/v3/projects/fd3bad6f36d44fc5b9c39b1a130c2a38` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:01.824905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fd3bad6f36d44fc5b9c39b1a130c2a38)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:01.832168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:01.855728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 857/1736] 10.222.0.65 () {58 vars in 1015 bytes} [Tue Sep 13 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:01.989101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3bad6f36d44fc5b9c39b1a130c2a38', 'request_id': 'req-4de65fd6-0712-43ea-af8a-e89e7aaa6461'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:01.996998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:02.000585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:02.001314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3bad6f36d44fc5b9c39b1a130c2a38', 'request_id': 'req-4de65fd6-0712-43ea-af8a-e89e7aaa6461'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:02.007608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:02.012409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3bad6f36d44fc5b9c39b1a130c2a38', 'request_id': 'req-4de65fd6-0712-43ea-af8a-e89e7aaa6461'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:02.037507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] The token cache is being invalidate because project fd3bad6f36d44fc5b9c39b1a130c2a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:02.037808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:02.043333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ce01-6080-47be-83e6-e1185c6248fa None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:02.053182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4de65fd6-0712-43ea-af8a-e89e7aaa6461 tempest-VolumeQuotaClassesTest-116653260 tempest-VolumeQuotaClassesTest-116653260-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:02.055197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 880/1737] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:01 2022] DELETE /identity/v3/projects/fd3bad6f36d44fc5b9c39b1a130c2a38 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:02.379489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c984-a411-4603-8ef8-41b84c3bfd13 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:02.380955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c984-a411-4603-8ef8-41b84c3bfd13 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:02.381472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c984-a411-4603-8ef8-41b84c3bfd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:02.388039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:02.395927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:02.397254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:02.399368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:02.409434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:02.419004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c1f6c984-a411-4603-8ef8-41b84c3bfd13 None None] MFA Rules not processed for user `0625b3113ce044bbac4f6142eaade11c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:02.420327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c1f6c984-a411-4603-8ef8-41b84c3bfd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:02.490095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:02.494076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:02.595610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 858/1738] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:02.641328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 881/1739] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:02.665211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:02.666919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:02.886742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dd419-aac5-404d-8a27-d27272ca574d service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:02.930694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:02.931276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:02.931611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:02.937783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] PATH_INFO: `/v3/users/274ed1b7000448e8bce85786d359f281` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:02.943653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=274ed1b7000448e8bce85786d359f281)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:02.950766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:03.113488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274ed1b7000448e8bce85786d359f281'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:03.139840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274ed1b7000448e8bce85786d359f281'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:03.141541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274ed1b7000448e8bce85786d359f281'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:03.181198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274ed1b7000448e8bce85786d359f281'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:03.271756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274ed1b7000448e8bce85786d359f281'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:03.293686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 859/1740] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:02 2022] DELETE /identity/v3/users/274ed1b7000448e8bce85786d359f281 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:03.299065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:03.307973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:03.307973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:03.365705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:03.366840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:03.646333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 882/1741] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:03.742270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:03.744606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:03.760386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:03.887421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897711a-34f6-4e3d-b774-226f9056bd4a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:03.902057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:03.902838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:03.903138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:03.903404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:03.923619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:03.925382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:03.958686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 860/1742] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:04.049113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:04.050979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:04.070333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f3bea-ee4f-4521-9c07-c5ad11b37a78 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:04.088637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:04.089720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:04.090238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:04.090786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] PATH_INFO: `/v3/projects/4a9bc1ff5f234d1db581da768a84b07e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:04.095501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9bc1ff5f234d1db581da768a84b07e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:04.097854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:04.275161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9bc1ff5f234d1db581da768a84b07e', 'request_id': 'req-7f72c830-d84f-4e80-baa7-f60e5fecc139'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.293164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9bc1ff5f234d1db581da768a84b07e', 'request_id': 'req-7f72c830-d84f-4e80-baa7-f60e5fecc139'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.314714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9bc1ff5f234d1db581da768a84b07e', 'request_id': 'req-7f72c830-d84f-4e80-baa7-f60e5fecc139'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.353526 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] The token cache is being invalidate because project 4a9bc1ff5f234d1db581da768a84b07e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:04.354698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.356869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.361234 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 883/1743] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:04 2022] DELETE /identity/v3/projects/4a9bc1ff5f234d1db581da768a84b07e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:04.365635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:04.382288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a246d80-68a4-40c1-9a70-e858a504dddc service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:04.412973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:04.414654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:04.482191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:04.565019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:04.565755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:04.566181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:04.566537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] PATH_INFO: `/v3/users/154a868e40434399b436dc3f0fb6d081` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:04.567857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=154a868e40434399b436dc3f0fb6d081)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:04.570755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:04.703323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154a868e40434399b436dc3f0fb6d081'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.761254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154a868e40434399b436dc3f0fb6d081'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.767610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154a868e40434399b436dc3f0fb6d081'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.777578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72c830-d84f-4e80-baa7-f60e5fecc139 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:04.796588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:04.797451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:04.797874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:04.798423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] PATH_INFO: `/v3/projects/56bd882fb4e44545b630d8d880f0d030` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:04.800332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56bd882fb4e44545b630d8d880f0d030)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:04.804170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:04.818613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154a868e40434399b436dc3f0fb6d081'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.879562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154a868e40434399b436dc3f0fb6d081'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:04.909380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 861/1744] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:04 2022] DELETE /identity/v3/users/154a868e40434399b436dc3f0fb6d081 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:05.039455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56bd882fb4e44545b630d8d880f0d030', 'request_id': 'req-3bf986c3-c19c-4a7d-95f3-aa577d327925'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.054695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56bd882fb4e44545b630d8d880f0d030', 'request_id': 'req-3bf986c3-c19c-4a7d-95f3-aa577d327925'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.071441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56bd882fb4e44545b630d8d880f0d030', 'request_id': 'req-3bf986c3-c19c-4a7d-95f3-aa577d327925'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.111815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] The token cache is being invalidate because project 56bd882fb4e44545b630d8d880f0d030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:05.112118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.114755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.139555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 884/1745] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:04 2022] DELETE /identity/v3/projects/56bd882fb4e44545b630d8d880f0d030 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:05.141334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:05.143107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:05.232516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:05.357667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef520d0d-9caf-47e1-b219-1cb48b3659d9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:05.582263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:05.582892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:05.585637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:05.586243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] PATH_INFO: `/v3/users/6908e0ecf87043cc8c6171c37e702b58` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:05.589373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6908e0ecf87043cc8c6171c37e702b58)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:05.597144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:05.722867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6908e0ecf87043cc8c6171c37e702b58'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.745517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6908e0ecf87043cc8c6171c37e702b58'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.746330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6908e0ecf87043cc8c6171c37e702b58'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.758055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6908e0ecf87043cc8c6171c37e702b58'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.788751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6908e0ecf87043cc8c6171c37e702b58'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:05.801358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 862/1746] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:05 2022] DELETE /identity/v3/users/6908e0ecf87043cc8c6171c37e702b58 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:07.617309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:07.619636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:07.665696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf986c3-c19c-4a7d-95f3-aa577d327925 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:07.886041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:07.887209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:07.888686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:07.889113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] PATH_INFO: `/v3/projects/3424d6aa3dc34ab8bf8b255ac906736d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:07.891190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3424d6aa3dc34ab8bf8b255ac906736d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:07.903797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:08.101574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3424d6aa3dc34ab8bf8b255ac906736d', 'request_id': 'req-a41a1c21-d980-4e65-a31b-20fd10e94ebb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.117839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:08.143680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3424d6aa3dc34ab8bf8b255ac906736d', 'request_id': 'req-a41a1c21-d980-4e65-a31b-20fd10e94ebb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.181985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3424d6aa3dc34ab8bf8b255ac906736d', 'request_id': 'req-a41a1c21-d980-4e65-a31b-20fd10e94ebb'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.221247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] The token cache is being invalidate because project 3424d6aa3dc34ab8bf8b255ac906736d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:08.221554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.232142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a41a1c21-d980-4e65-a31b-20fd10e94ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.239633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 885/1747] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:07 2022] DELETE /identity/v3/projects/3424d6aa3dc34ab8bf8b255ac906736d => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:08.241206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:08.243379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:08.298170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3989b39-4196-48e7-8f65-b8f4d2153ab9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:08.386126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8e33b-a30c-474c-a057-8f505846c381 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:08.386421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8e33b-a30c-474c-a057-8f505846c381 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:08.386826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8e33b-a30c-474c-a057-8f505846c381 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:08.388487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 886/1748] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:02:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 18:02:08.595054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:08.637475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:08.639618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:08.640126 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:08.640750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] PATH_INFO: `/v3/users/ff13187ee695496f86064c673a5391d7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:08.643802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff13187ee695496f86064c673a5391d7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:08.649376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:08.822664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff13187ee695496f86064c673a5391d7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.827770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:08.829007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:08.829448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:08.857015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff13187ee695496f86064c673a5391d7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.858017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff13187ee695496f86064c673a5391d7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.879184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff13187ee695496f86064c673a5391d7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.917638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:08.927570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:08.935559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff13187ee695496f86064c673a5391d7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:08.967701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 863/1749] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:08 2022] DELETE /identity/v3/users/ff13187ee695496f86064c673a5391d7 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:09.129604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:09.131407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:09.227279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 887/1750] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:02:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 13 18:02:09.385194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672dbdd-77c7-4755-9ced-f58ade2409a2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:09.415355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:09.418960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:09.420114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:09.421620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] PATH_INFO: `/v3/projects/8c09a83758bd49dcb5f14cd0ce7f14d3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:09.425277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c09a83758bd49dcb5f14cd0ce7f14d3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:09.432973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:09.521448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:09.523373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:09.561189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc88e8e-d10f-4563-9818-d90222fdd324 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:09.654547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c09a83758bd49dcb5f14cd0ce7f14d3', 'request_id': 'req-0e57faf4-131f-47a0-8af1-b3a657ae9d48'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:09.701362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c09a83758bd49dcb5f14cd0ce7f14d3', 'request_id': 'req-0e57faf4-131f-47a0-8af1-b3a657ae9d48'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:09.721744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c09a83758bd49dcb5f14cd0ce7f14d3', 'request_id': 'req-0e57faf4-131f-47a0-8af1-b3a657ae9d48'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:09.759148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] The token cache is being invalidate because project 8c09a83758bd49dcb5f14cd0ce7f14d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:09.760306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:09.762260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0e57faf4-131f-47a0-8af1-b3a657ae9d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:09.768632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 864/1751] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:09 2022] DELETE /identity/v3/projects/8c09a83758bd49dcb5f14cd0ce7f14d3 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:09.784234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606473a-7e54-4c53-b1af-0d373f036c72 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:09.784881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606473a-7e54-4c53-b1af-0d373f036c72 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:09.785366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606473a-7e54-4c53-b1af-0d373f036c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:09.820260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:09.855088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b369fc-d13f-4fb4-8513-dc6a572ffc02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:09.858137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b369fc-d13f-4fb4-8513-dc6a572ffc02 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:09.858722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b369fc-d13f-4fb4-8513-dc6a572ffc02 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:09.860055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b369fc-d13f-4fb4-8513-dc6a572ffc02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:09.866248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d0b369fc-d13f-4fb4-8513-dc6a572ffc02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:09.867711 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6606473a-7e54-4c53-b1af-0d373f036c72 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:09.872317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6606473a-7e54-4c53-b1af-0d373f036c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:09.914998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b369fc-d13f-4fb4-8513-dc6a572ffc02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:09.917398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b369fc-d13f-4fb4-8513-dc6a572ffc02 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:09.946511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 888/1752] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:10.040321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 865/1753] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:10.051030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6a48-f704-4fe1-a0fe-b3e571eb86a4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.051895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6a48-f704-4fe1-a0fe-b3e571eb86a4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.053643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a6a48-f704-4fe1-a0fe-b3e571eb86a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.087288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a61a6a48-f704-4fe1-a0fe-b3e571eb86a4 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:10.089009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a61a6a48-f704-4fe1-a0fe-b3e571eb86a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:10.123112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 889/1754] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:10.128039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606473a-7e54-4c53-b1af-0d373f036c72 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:10.140784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606473a-7e54-4c53-b1af-0d373f036c72 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:10.156588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:10.158114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.158675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.159420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.160380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:10.161038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:10.163014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:10.186206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 866/1755] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:10.187821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6a48-f704-4fe1-a0fe-b3e571eb86a4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:10.201152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a6a48-f704-4fe1-a0fe-b3e571eb86a4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:10.207399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:10.208406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.209067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.209574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.210484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:10.213043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:10.269178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:10.271650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 890/1756] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:10 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:10.285364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e1060-067e-45b6-bb5e-a5862d450553 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:10.381976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:10.383003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.383500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.384005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.384815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:10.386973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:10.440233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:10.453189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 867/1757] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:10 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:10.457261 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ef35a-bc0c-4f77-9b48-0d0dc0e03757 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:10.463255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:10.464083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.464537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.464918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.467607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:10.469514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:10.488168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:10.495515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 891/1758] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:10.501585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf8a5f-2a83-48a0-beee-b6d3a1f105c9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:10.506522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a5be2-b430-4266-8016-d71af1d67ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:10.509743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a5be2-b430-4266-8016-d71af1d67ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.510027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a5be2-b430-4266-8016-d71af1d67ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.510379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a5be2-b430-4266-8016-d71af1d67ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.511551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a5be2-b430-4266-8016-d71af1d67ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:10.513073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a5be2-b430-4266-8016-d71af1d67ad1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:10.560982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 868/1759] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:10.561309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:10.581161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb0fc2-03bc-4070-8d3b-edfc7d0d8401 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:10.602860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:10.603607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.603843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.604112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] PATH_INFO: `/v3/projects/62b9c9dd59d94b3a9787e460a0f4c2fc/users/ac17c310ced14258a9254e45298d1f9d/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.646028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:10.681372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b9c9dd59d94b3a9787e460a0f4c2fc, user_id=ac17c310ced14258a9254e45298d1f9d, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:10.684682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:10.744306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 892/1760] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:10 2022] PUT /identity/v3/projects/62b9c9dd59d94b3a9787e460a0f4c2fc/users/ac17c310ced14258a9254e45298d1f9d/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:10.756099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62fff5-6f61-4665-9c0d-9109520c4e04 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:10.756465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62fff5-6f61-4665-9c0d-9109520c4e04 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:10.756805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62fff5-6f61-4665-9c0d-9109520c4e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:10.829082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6d62fff5-6f61-4665-9c0d-9109520c4e04 None None] MFA Rules not processed for user `ac17c310ced14258a9254e45298d1f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:10.830384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6d62fff5-6f61-4665-9c0d-9109520c4e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:10.891841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:10.893913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:10.980987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:11.010752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 869/1761] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:11.020245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62fff5-6f61-4665-9c0d-9109520c4e04 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.102874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0cb48-2e9e-4532-b89e-f46c907271e5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.127843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.129246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.129626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.130704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] PATH_INFO: `/v3/projects/5f3d7c67967844b48550a0b221283464` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.133945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3d7c67967844b48550a0b221283464)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.141094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.254066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62fff5-6f61-4665-9c0d-9109520c4e04 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.261679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.262231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.262507 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.262781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.263613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.266979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.313848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 870/1762] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:11 2022] POST /identity/v3/projects => generated 383 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:11.316095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3d7c67967844b48550a0b221283464', 'request_id': 'req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:11.317912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.329692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3d7c67967844b48550a0b221283464', 'request_id': 'req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:11.343605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3d7c67967844b48550a0b221283464', 'request_id': 'req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:11.373375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] The token cache is being invalidate because project 5f3d7c67967844b48550a0b221283464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:11.373687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:11.378469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75079ae8-7ad9-49e9-8bf5-f7cbb04517a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:11.380828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 893/1763] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:10 2022] DELETE /identity/v3/projects/5f3d7c67967844b48550a0b221283464 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:11.385292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cf4f0-3803-4740-9ae5-3c086cd5a2cb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.385721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cf4f0-3803-4740-9ae5-3c086cd5a2cb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.385863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cf4f0-3803-4740-9ae5-3c086cd5a2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.433458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bf7cf4f0-3803-4740-9ae5-3c086cd5a2cb None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:11.435052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bf7cf4f0-3803-4740-9ae5-3c086cd5a2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:11.455982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 894/1764] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:11.459139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ae98a-2eaa-42a9-b8b9-4171e3e9c716 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.464489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca649d21-d0aa-443d-adc7-bc3cd057e0f7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.464794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca649d21-d0aa-443d-adc7-bc3cd057e0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.465255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca649d21-d0aa-443d-adc7-bc3cd057e0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.467895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.468958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.469423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.470036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.473790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.476099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.510797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 871/1765] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:11 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:11.510975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ca649d21-d0aa-443d-adc7-bc3cd057e0f7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:11.512144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.513655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ca649d21-d0aa-443d-adc7-bc3cd057e0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:11.514129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:11.531661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 895/1766] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:11.532859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea05d8a-4c65-4ae2-ad98-d58ca64b1938 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.535578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca649d21-d0aa-443d-adc7-bc3cd057e0f7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.539617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.540313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.540627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.542580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.543139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.546255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.555531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca649d21-d0aa-443d-adc7-bc3cd057e0f7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.579339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 872/1767] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:11.580941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.581229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.581518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.581798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.582474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:11.582828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.584467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.585788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.596999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf4eb1-bacc-436b-bdef-bc2c0875dc3e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.604119 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.605096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.605724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.606190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] PATH_INFO: `/v3/projects/241617753e144418a0c09e3804dee2cc/users/a37b27a27e8842458009384e42b8bde9/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.614118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 896/1768] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:11.615288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.626703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7040a56-3fc2-4768-a025-24966321da60 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.628674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241617753e144418a0c09e3804dee2cc, user_id=a37b27a27e8842458009384e42b8bde9, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.631790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.635188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.635821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.636104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.636377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.637066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.639156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.656868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 873/1769] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:11 2022] PUT /identity/v3/projects/241617753e144418a0c09e3804dee2cc/users/a37b27a27e8842458009384e42b8bde9/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:11.661398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.684089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 897/1770] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:11 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:11.685316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.734503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:11.778028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:11.825372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccf22f-0d93-4b38-89a8-3e7a2075a016 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.830139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ed0f1-ce4c-47de-b350-f9742d694803 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.839320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.840160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.840471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.840779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.841460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.846485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.857848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.857848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.857848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.857848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.858286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.858286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.884856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 874/1771] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:11.886431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.887649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:11.893742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 898/1772] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:11 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:11.895955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:11.911092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088f053-aefb-42e4-a8a1-59d58ef25ea4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.913062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72febf2-7fd4-41e9-a24d-f4457133faea admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:11.917823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.918784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.919758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.920664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] PATH_INFO: `/v3/projects/241617753e144418a0c09e3804dee2cc/users/a37b27a27e8842458009384e42b8bde9/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.925605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241617753e144418a0c09e3804dee2cc, user_id=a37b27a27e8842458009384e42b8bde9, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.929521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.930452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:11.931114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:11.931395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:11.931669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] PATH_INFO: `/v3/projects/8d55c3565cb24c9db82d46fc9b8dd404` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:11.932876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d55c3565cb24c9db82d46fc9b8dd404)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:11.934818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:11.963726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 899/1773] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:11 2022] PUT /identity/v3/projects/241617753e144418a0c09e3804dee2cc/users/a37b27a27e8842458009384e42b8bde9/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:11.965434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:12.043767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d55c3565cb24c9db82d46fc9b8dd404', 'request_id': 'req-4d44e828-bb8d-451c-a2e7-7d717c8b622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:12.056030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d55c3565cb24c9db82d46fc9b8dd404', 'request_id': 'req-4d44e828-bb8d-451c-a2e7-7d717c8b622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:12.074565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d55c3565cb24c9db82d46fc9b8dd404', 'request_id': 'req-4d44e828-bb8d-451c-a2e7-7d717c8b622a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:12.089296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] The token cache is being invalidate because project 8d55c3565cb24c9db82d46fc9b8dd404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:12.090001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:12.091038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4d44e828-bb8d-451c-a2e7-7d717c8b622a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:12.102141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 875/1774] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:11 2022] DELETE /identity/v3/projects/8d55c3565cb24c9db82d46fc9b8dd404 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:12.103511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.103794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.104062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.123710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db389-aedd-4578-a877-5b7470fd25f1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:12.162502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] MFA Rules not processed for user `a37b27a27e8842458009384e42b8bde9`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:12.163473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:12.245151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72a3ac-04e0-496c-b9fe-bc474016dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:12.246229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72a3ac-04e0-496c-b9fe-bc474016dd96 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.246788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72a3ac-04e0-496c-b9fe-bc474016dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.247319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72a3ac-04e0-496c-b9fe-bc474016dd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.248295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72a3ac-04e0-496c-b9fe-bc474016dd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:12.267874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72a3ac-04e0-496c-b9fe-bc474016dd96 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:12.294484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 900/1775] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:12.294484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.294484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.294484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.314068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 876/1776] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:12.314992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:12.325368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:12.332305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:12.335040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:12.349145 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c06d63-2fe2-48e3-8724-5c4f3673b522 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:12.357836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76cd8-4401-4c93-817b-790447c6b0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:12.358482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76cd8-4401-4c93-817b-790447c6b0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.358772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76cd8-4401-4c93-817b-790447c6b0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.358905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 901/1777] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:12.359037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76cd8-4401-4c93-817b-790447c6b0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.359707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76cd8-4401-4c93-817b-790447c6b0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:12.359987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:12.361460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76cd8-4401-4c93-817b-790447c6b0a2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:12.362372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:12.388316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 877/1778] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:12.389617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cbbdd-0903-454a-8aef-f9a482bc9c04 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.389911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cbbdd-0903-454a-8aef-f9a482bc9c04 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.390267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86cbbdd-0903-454a-8aef-f9a482bc9c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.437521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b86cbbdd-0903-454a-8aef-f9a482bc9c04 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:12.438822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b86cbbdd-0903-454a-8aef-f9a482bc9c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:12.458943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 878/1779] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:12.459802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cbbdd-0903-454a-8aef-f9a482bc9c04 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:12.467840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:12.481669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86cbbdd-0903-454a-8aef-f9a482bc9c04 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:12.482663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa61dc-7415-41b4-adf4-a5e295ea0534 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:12.496053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:12.497703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:12.498339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.499543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.500234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.500545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.500973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.501956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] PATH_INFO: `/v3/projects/3119ffe3f67d48e9b994463590d95f6c/users/05905557e1734182bfed315366345c77/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.505177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:12.573482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3119ffe3f67d48e9b994463590d95f6c, user_id=05905557e1734182bfed315366345c77, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:12.580146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:12.612376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 879/1780] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:12 2022] PUT /identity/v3/projects/3119ffe3f67d48e9b994463590d95f6c/users/05905557e1734182bfed315366345c77/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:12.621976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:12.625680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:12.762247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf17d2e-a86f-4577-ac1f-f5b5fb06d358 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:12.770295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c27293-8f54-4782-bf20-44cb1a78ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:12.772197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27293-8f54-4782-bf20-44cb1a78ac9c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.772548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27293-8f54-4782-bf20-44cb1a78ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.772950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c27293-8f54-4782-bf20-44cb1a78ac9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.773617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c27293-8f54-4782-bf20-44cb1a78ac9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:12.773921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c27293-8f54-4782-bf20-44cb1a78ac9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:12.775948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c27293-8f54-4782-bf20-44cb1a78ac9c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:12.796303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 880/1781] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:12.796303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66309dcf-a811-433a-93bb-0bdb4ecd1eaf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.796303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66309dcf-a811-433a-93bb-0bdb4ecd1eaf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.796303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66309dcf-a811-433a-93bb-0bdb4ecd1eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.812919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:12.814445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:12.842703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 902/1782] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:12.843403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:12.851247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-66309dcf-a811-433a-93bb-0bdb4ecd1eaf None None] MFA Rules not processed for user `05905557e1734182bfed315366345c77`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:12.853073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-66309dcf-a811-433a-93bb-0bdb4ecd1eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:12.880989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccb2b4-58b7-42f2-90cb-8e18fb0966fe service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:12.897851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:12.898520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.898757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.899031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.905703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:12.908027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:12.940485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 881/1783] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:12.943454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66309dcf-a811-433a-93bb-0bdb4ecd1eaf None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:12.961294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66309dcf-a811-433a-93bb-0bdb4ecd1eaf None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:12.968285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:12.969233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:12.969838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:12.970319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:12.971330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:12.974413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:12.978147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 903/1784] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:12 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:12.982358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.027605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 882/1785] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:12 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:13.033187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.180435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349f479-5000-4f06-87c7-31d4de81d43b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.283013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00640e78-3903-4f50-b3a7-e855414b2c11 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.295469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.298111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.299189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.301484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.302298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.309760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.347902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:13.376842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 883/1786] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:13 2022] POST /identity/v3/users => generated 295 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:13.381701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.395047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.395906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.396265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.396574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.397480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.405406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3821cb-dd66-4aa0-8f32-6fb4e56cb267 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.406401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.416759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.418801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.420175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.420581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.421921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.426350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.465178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 884/1787] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:13.467086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.487343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 904/1788] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:12 2022] POST /identity/v3/users => generated 290 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:13.490523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.492338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228434c8-d995-434f-a6e4-a5d336e7e9d3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.500466 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.501483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.501858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.502224 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] PATH_INFO: `/v3/projects/71d6a02b7bfa459e838d8d76c061ccc9/users/60c823809e1d466083f790b779143c06/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.523765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fad3ae-7fcc-4abc-afea-2e8e644a52e4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.537747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.539089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.539428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.539531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d6a02b7bfa459e838d8d76c061ccc9, user_id=60c823809e1d466083f790b779143c06, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.539707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.540591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.542456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.542692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.569830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 905/1789] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:13.570974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.580251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:13.582538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 885/1790] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:13 2022] PUT /identity/v3/projects/71d6a02b7bfa459e838d8d76c061ccc9/users/60c823809e1d466083f790b779143c06/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:13.586858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.589929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51c827-418c-4c81-8e3b-8180dbe9ecfd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.754337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd0411-3313-4fa3-9d2a-dc1f5b00948c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.765200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.769018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.769340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.769812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.770614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.771563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.773488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.775199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.776281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.777396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.796865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.816172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 886/1791] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:13.818141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.823818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.850691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 906/1792] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:13.856382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a942fd4-e42b-486c-a179-56eb794517ff admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.866904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e07ba-18ae-4311-830c-34ef887fd2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:13.867607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e07ba-18ae-4311-830c-34ef887fd2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.867908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e07ba-18ae-4311-830c-34ef887fd2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.868178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e07ba-18ae-4311-830c-34ef887fd2e9 admin admin] PATH_INFO: `/v3/projects/71d6a02b7bfa459e838d8d76c061ccc9/users/60c823809e1d466083f790b779143c06/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.872035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:13.873504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e07ba-18ae-4311-830c-34ef887fd2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d6a02b7bfa459e838d8d76c061ccc9, user_id=60c823809e1d466083f790b779143c06, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:13.877383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e07ba-18ae-4311-830c-34ef887fd2e9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:13.911119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce2069-fa42-48b0-ba7a-4f3388c89aed admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:13.914341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 887/1793] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:13 2022] PUT /identity/v3/projects/71d6a02b7bfa459e838d8d76c061ccc9/users/60c823809e1d466083f790b779143c06/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:13.925510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:13.925797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:13.926076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:13.976133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] MFA Rules not processed for user `60c823809e1d466083f790b779143c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:13.978881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:14.058980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd34eb-68ce-4f6e-a2aa-55ebe53d31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:14.059954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd34eb-68ce-4f6e-a2aa-55ebe53d31ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:14.060238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd34eb-68ce-4f6e-a2aa-55ebe53d31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:14.060666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd34eb-68ce-4f6e-a2aa-55ebe53d31ea admin admin] PATH_INFO: `/v3/projects/a9556cfb254c488a9c992c475b2a99f1/users/edcc2cbce42b477294ee77eabcb374d6/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:14.099891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd34eb-68ce-4f6e-a2aa-55ebe53d31ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9556cfb254c488a9c992c475b2a99f1, user_id=edcc2cbce42b477294ee77eabcb374d6, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:14.105435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd34eb-68ce-4f6e-a2aa-55ebe53d31ea admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:14.149220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 907/1794] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:13 2022] PUT /identity/v3/projects/a9556cfb254c488a9c992c475b2a99f1/users/edcc2cbce42b477294ee77eabcb374d6/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:14.155065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daf6d3-101e-4b68-9a25-44e47391da2c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:14.155363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daf6d3-101e-4b68-9a25-44e47391da2c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:14.155640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daf6d3-101e-4b68-9a25-44e47391da2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:14.172797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 888/1795] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:14.189815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:14.192754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:14.229822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-94daf6d3-101e-4b68-9a25-44e47391da2c None None] MFA Rules not processed for user `edcc2cbce42b477294ee77eabcb374d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:14.231499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-94daf6d3-101e-4b68-9a25-44e47391da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:14.264604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:14.309974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074933ce-32c2-4dcc-b9cd-f3dcb46a5cdc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:14.400201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 908/1796] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:14.411107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:14.412096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:14.412522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94daf6d3-101e-4b68-9a25-44e47391da2c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:14.412883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:14.413373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:14.433209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:14.588044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94daf6d3-101e-4b68-9a25-44e47391da2c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:14.596442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760d9f3-ecde-43e7-92a9-539208b88ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:14.597557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d9f3-ecde-43e7-92a9-539208b88ffd admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:14.600317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d9f3-ecde-43e7-92a9-539208b88ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:14.600317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760d9f3-ecde-43e7-92a9-539208b88ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:14.600317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760d9f3-ecde-43e7-92a9-539208b88ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:14.604253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760d9f3-ecde-43e7-92a9-539208b88ffd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:14.622452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:14.624048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:14.645763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 889/1797] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:14.672091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 909/1798] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:14 2022] POST /identity/v3/projects => generated 379 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:14.676588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:14.719858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ee79c-759a-4077-8b1c-4ff015e5e6e1 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:14.733240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a730341-d0f1-4366-a70e-7c5342bb3aac None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:14.733803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a730341-d0f1-4366-a70e-7c5342bb3aac None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:14.734253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a730341-d0f1-4366-a70e-7c5342bb3aac None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:14.736661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 910/1799] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:02:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 18:02:14.772374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d86d9-5900-4da3-81ba-6b624276b504 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:14.772729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d86d9-5900-4da3-81ba-6b624276b504 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:14.773007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d86d9-5900-4da3-81ba-6b624276b504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:14.845124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-df0d86d9-5900-4da3-81ba-6b624276b504 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:14.849946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-df0d86d9-5900-4da3-81ba-6b624276b504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:14.935043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:14.935676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:14.937844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:14.938310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:14.938980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:14.942405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:14.992073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 890/1800] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:14 2022] POST /identity/v3/users => generated 290 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:14.996335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:15.024554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ffe3b-0c48-4465-aa9b-1e96557bae09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:15.037225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 911/1801] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:02:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:15.038575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:15.039215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.039566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.039841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.040535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:15.042187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:15.062998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 891/1802] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:15.067170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d86d9-5900-4da3-81ba-6b624276b504 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:15.087388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d86d9-5900-4da3-81ba-6b624276b504 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:15.106257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23908e-fe77-4635-9dbe-5da93c73df38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:15.112198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23908e-fe77-4635-9dbe-5da93c73df38 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.113081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23908e-fe77-4635-9dbe-5da93c73df38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.116605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23908e-fe77-4635-9dbe-5da93c73df38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.117327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23908e-fe77-4635-9dbe-5da93c73df38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:15.123433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23908e-fe77-4635-9dbe-5da93c73df38 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:15.158699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 912/1803] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:15.163225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:15.180029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad3c12-90b0-4f26-8b75-ad08672643a4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:15.201841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:15.203470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.203820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.204109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] PATH_INFO: `/v3/projects/b38fd723a8e44dda98f863fd0dd75d8e/users/e957026b99ad4fd198beac2a5b3e60c6/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.228560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:15.247806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38fd723a8e44dda98f863fd0dd75d8e, user_id=e957026b99ad4fd198beac2a5b3e60c6, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:15.254195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:15.308816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 892/1804] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:15 2022] PUT /identity/v3/projects/b38fd723a8e44dda98f863fd0dd75d8e/users/e957026b99ad4fd198beac2a5b3e60c6/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:15.326084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7e08-47e4-4ce0-9b69-93547d41a9b2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.326408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7e08-47e4-4ce0-9b69-93547d41a9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.326697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b7e08-47e4-4ce0-9b69-93547d41a9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.381475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9e4b7e08-47e4-4ce0-9b69-93547d41a9b2 None None] MFA Rules not processed for user `e957026b99ad4fd198beac2a5b3e60c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:15.382865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9e4b7e08-47e4-4ce0-9b69-93547d41a9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:15.448905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:15.451849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:15.543507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 913/1805] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:15.548832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7e08-47e4-4ce0-9b69-93547d41a9b2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:15.568825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:15.612569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941d64a-6770-4bc2-afa7-a13bfbe8875d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:15.632078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:15.633264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.633372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.636648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.644712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b7e08-47e4-4ce0-9b69-93547d41a9b2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:15.651373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:15.654655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.654940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.655218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.656565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:15.659544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:15.661674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:15.671826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:15.706915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 893/1806] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:15.722420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 914/1807] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:15 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:15.724223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:15.743596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805499a-e5af-4ffb-ad5b-1f18eeecf6cb service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:15.888175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:15.889091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.889529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.889956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.890787 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:15.893631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:15.953499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 894/1808] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:15 2022] POST /identity/v3/users => generated 296 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:15.955798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:15.975083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfec8c-b814-4a8e-90cf-0ae878cc76e9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:15.992087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:15.992782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:15.993064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:15.993379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:15.994080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:15.995711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:16.014317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 915/1809] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:16.018252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:16.029765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb97cc-c928-4eaf-85af-386d7ef633ce admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:16.035304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:16.036311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:16.036441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:16.036682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] PATH_INFO: `/v3/projects/1438df939f6d4819a80fb4c10b0c690a/users/0dafa233cc984227b8af646967fff387/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:16.060296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1438df939f6d4819a80fb4c10b0c690a, user_id=0dafa233cc984227b8af646967fff387, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:16.063034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:16.077573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 895/1810] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:16 2022] PUT /identity/v3/projects/1438df939f6d4819a80fb4c10b0c690a/users/0dafa233cc984227b8af646967fff387/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:16.081754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:16.096237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d248a55-d06a-4b05-88bc-9d091f3852a6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:16.168722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da822e5b-76b0-468d-aeb8-3ed0d0f6f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:16.169350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da822e5b-76b0-468d-aeb8-3ed0d0f6f34f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:16.169630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da822e5b-76b0-468d-aeb8-3ed0d0f6f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:16.169902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da822e5b-76b0-468d-aeb8-3ed0d0f6f34f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:16.170558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da822e5b-76b0-468d-aeb8-3ed0d0f6f34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:16.172178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da822e5b-76b0-468d-aeb8-3ed0d0f6f34f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:16.183312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 916/1811] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:16.187822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:16.199056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea95a1-d7ae-4f41-b3cf-cddfe588d8bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:16.204271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:16.205125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:16.205403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:16.205674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] PATH_INFO: `/v3/projects/1438df939f6d4819a80fb4c10b0c690a/users/0dafa233cc984227b8af646967fff387/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:16.207649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1438df939f6d4819a80fb4c10b0c690a, user_id=0dafa233cc984227b8af646967fff387, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:16.210433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:16.225823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 896/1812] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:16 2022] PUT /identity/v3/projects/1438df939f6d4819a80fb4c10b0c690a/users/0dafa233cc984227b8af646967fff387/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:16.236034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b87e5-5d28-49e0-8df6-54845b9406fa None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:16.236395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b87e5-5d28-49e0-8df6-54845b9406fa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:16.236711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b87e5-5d28-49e0-8df6-54845b9406fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:16.266425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-141b87e5-5d28-49e0-8df6-54845b9406fa None None] MFA Rules not processed for user `0dafa233cc984227b8af646967fff387`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:16.267654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-141b87e5-5d28-49e0-8df6-54845b9406fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:16.349808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 917/1813] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:16.362766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:16.401013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:16.438318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0972d3-4162-4622-a885-dc96f7c0f7bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:16.442901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60a2f-8535-4d7c-8cc1-1dcfdb0ccafa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:16.443531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60a2f-8535-4d7c-8cc1-1dcfdb0ccafa service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:16.443805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60a2f-8535-4d7c-8cc1-1dcfdb0ccafa service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:16.444136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60a2f-8535-4d7c-8cc1-1dcfdb0ccafa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:16.455017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60a2f-8535-4d7c-8cc1-1dcfdb0ccafa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:16.456512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba60a2f-8535-4d7c-8cc1-1dcfdb0ccafa service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:16.472407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 897/1814] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:16.533055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b87e5-5d28-49e0-8df6-54845b9406fa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:16.547941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:16.552930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b87e5-5d28-49e0-8df6-54845b9406fa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:16.558253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17951459-458b-41b4-9b61-6c25852f9473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:16.559213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951459-458b-41b4-9b61-6c25852f9473 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:16.559651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951459-458b-41b4-9b61-6c25852f9473 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:16.559983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951459-458b-41b4-9b61-6c25852f9473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:16.625224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17951459-458b-41b4-9b61-6c25852f9473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:16.626701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17951459-458b-41b4-9b61-6c25852f9473 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:16.640695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 918/1815] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:24.853888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31314c-849a-4026-b274-0ffda179d12a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:24.854473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31314c-849a-4026-b274-0ffda179d12a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:24.854875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31314c-849a-4026-b274-0ffda179d12a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:24.857444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 898/1816] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:02:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 18:02:24.870969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e9a01-b1e8-4041-962a-1dffbfb6710d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:24.872374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e9a01-b1e8-4041-962a-1dffbfb6710d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:24.872874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e9a01-b1e8-4041-962a-1dffbfb6710d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:24.922634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f68e9a01-b1e8-4041-962a-1dffbfb6710d None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:24.924331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f68e9a01-b1e8-4041-962a-1dffbfb6710d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:24.960788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31314c-849a-4026-b274-0ffda179d12a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:24.976794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31314c-849a-4026-b274-0ffda179d12a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:24.984907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:24.990623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:24.990623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:24.990915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:25.098039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 919/1817] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:02:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:25.104206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e9a01-b1e8-4041-962a-1dffbfb6710d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:25.133175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:25.134850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:25.156794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 899/1818] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:25.172681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:25.194623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651849c8-b5b3-4589-b0e9-ed8b13b34e44 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:25.199496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:25.200409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:25.200962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:25.201445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:25.204304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e9a01-b1e8-4041-962a-1dffbfb6710d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:25.209697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:25.211585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:25.211877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:25.212142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] PATH_INFO: `/v3/users/edcc2cbce42b477294ee77eabcb374d6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:25.213489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcc2cbce42b477294ee77eabcb374d6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:25.216188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:25.277265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:25.289901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:25.292666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:25.293306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcc2cbce42b477294ee77eabcb374d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:25.315728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 900/1819] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:25.317479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcc2cbce42b477294ee77eabcb374d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:25.318575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcc2cbce42b477294ee77eabcb374d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:25.324868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:25.340176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e69a8f-fa4b-427e-b8a6-555be98faf23 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:25.346863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcc2cbce42b477294ee77eabcb374d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:25.348963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:25.349812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:25.349928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:25.350253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:25.352826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:25.380853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:25.383901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:25.396631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7f5c7f40-4816-4936-a387-7d8735a0fb1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcc2cbce42b477294ee77eabcb374d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:25.411150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 901/1820] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:25.411738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 920/1821] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:25 2022] DELETE /identity/v3/users/edcc2cbce42b477294ee77eabcb374d6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:25.418212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:25.419371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:25.419713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:25.450347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:25.451840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:25.560221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 921/1822] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:25.623614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:25.625114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:25.816115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:25.832333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cf29-20a9-476c-b39a-5d869f5921da service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:25.944741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:25.945409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:25.945926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:25.946413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:25.960409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:25.962276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:25.981567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 902/1823] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:26.654567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:26.657799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:26.678856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786467-fcde-4e12-b17f-6b681b477e70 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:26.684652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:26.685450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:26.686240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:26.686240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] PATH_INFO: `/v3/projects/a9556cfb254c488a9c992c475b2a99f1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:26.689301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9556cfb254c488a9c992c475b2a99f1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:26.691806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:26.770681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9556cfb254c488a9c992c475b2a99f1', 'request_id': 'req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.780217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9556cfb254c488a9c992c475b2a99f1', 'request_id': 'req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.785890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9556cfb254c488a9c992c475b2a99f1', 'request_id': 'req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.797097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] The token cache is being invalidate because project a9556cfb254c488a9c992c475b2a99f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:26.797390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.798068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.801784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 922/1824] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:26 2022] DELETE /identity/v3/projects/a9556cfb254c488a9c992c475b2a99f1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:26.805671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:26.819053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977b25f-885e-4415-9132-6d7de9add200 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:26.890092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:26.891158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:26.891259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:26.891578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] PATH_INFO: `/v3/users/e957026b99ad4fd198beac2a5b3e60c6` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:26.892978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e957026b99ad4fd198beac2a5b3e60c6)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:26.895316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:26.951962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e957026b99ad4fd198beac2a5b3e60c6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.962762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e957026b99ad4fd198beac2a5b3e60c6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.963601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e957026b99ad4fd198beac2a5b3e60c6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.974194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e957026b99ad4fd198beac2a5b3e60c6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:26.997971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e957026b99ad4fd198beac2a5b3e60c6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.005836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 903/1825] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:26 2022] DELETE /identity/v3/users/e957026b99ad4fd198beac2a5b3e60c6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:27.494300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:27.495821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:27.509897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfbbc2-02b4-4161-be6a-1c142375ca9b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:27.552294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:27.580214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:27.580953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:27.581233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:27.581517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] PATH_INFO: `/v3/projects/b38fd723a8e44dda98f863fd0dd75d8e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:27.582656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38fd723a8e44dda98f863fd0dd75d8e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:27.584677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:27.636990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38fd723a8e44dda98f863fd0dd75d8e', 'request_id': 'req-4726d567-5f40-479a-8cee-b7664d76a650'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.646020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38fd723a8e44dda98f863fd0dd75d8e', 'request_id': 'req-4726d567-5f40-479a-8cee-b7664d76a650'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.652208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38fd723a8e44dda98f863fd0dd75d8e', 'request_id': 'req-4726d567-5f40-479a-8cee-b7664d76a650'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.663637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] The token cache is being invalidate because project b38fd723a8e44dda98f863fd0dd75d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:27.664339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.664584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.666077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 923/1826] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:27 2022] DELETE /identity/v3/projects/b38fd723a8e44dda98f863fd0dd75d8e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:27.671148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:27.673708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:27.688853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb594802-51f6-4848-b0f7-0cd5f0a0150f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:27.715290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:27.755881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:27.756561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:27.756838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:27.757108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] PATH_INFO: `/v3/users/0dafa233cc984227b8af646967fff387` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:27.758221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dafa233cc984227b8af646967fff387)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:27.760183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:27.816664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dafa233cc984227b8af646967fff387'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.825675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dafa233cc984227b8af646967fff387'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.826579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dafa233cc984227b8af646967fff387'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.834571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dafa233cc984227b8af646967fff387'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.846679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0c8a120e-9f66-418e-be31-34ab8809804d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dafa233cc984227b8af646967fff387'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:27.856540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 904/1827] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:27 2022] DELETE /identity/v3/users/0dafa233cc984227b8af646967fff387 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:28.160325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.161918 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:28.177401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726d567-5f40-479a-8cee-b7664d76a650 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:28.255050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:28.256107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.256107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.256418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] PATH_INFO: `/v3/projects/1438df939f6d4819a80fb4c10b0c690a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.257539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1438df939f6d4819a80fb4c10b0c690a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:28.260316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:28.318550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1438df939f6d4819a80fb4c10b0c690a', 'request_id': 'req-d70cc8ce-7d30-44bb-9819-4042e8b50751'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:28.325857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1438df939f6d4819a80fb4c10b0c690a', 'request_id': 'req-d70cc8ce-7d30-44bb-9819-4042e8b50751'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:28.331435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1438df939f6d4819a80fb4c10b0c690a', 'request_id': 'req-d70cc8ce-7d30-44bb-9819-4042e8b50751'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:28.339305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:28.345187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] The token cache is being invalidate because project 1438df939f6d4819a80fb4c10b0c690a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:28.345899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:28.346781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d70cc8ce-7d30-44bb-9819-4042e8b50751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:28.349653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 924/1828] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:28 2022] DELETE /identity/v3/projects/1438df939f6d4819a80fb4c10b0c690a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:28.357379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454315-d665-4fc4-9bac-de875e0b4b96 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.357379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454315-d665-4fc4-9bac-de875e0b4b96 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.357379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454315-d665-4fc4-9bac-de875e0b4b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.383390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-78454315-d665-4fc4-9bac-de875e0b4b96 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:28.384826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-78454315-d665-4fc4-9bac-de875e0b4b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:28.457383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 905/1829] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:28.466365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00d1a-373e-4d10-a430-b1d96250f4e1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.466578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00d1a-373e-4d10-a430-b1d96250f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.466854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00d1a-373e-4d10-a430-b1d96250f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.487092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8fe00d1a-373e-4d10-a430-b1d96250f4e1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:28.488298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8fe00d1a-373e-4d10-a430-b1d96250f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:28.496674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 925/1830] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:28.501164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78454315-d665-4fc4-9bac-de875e0b4b96 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.511354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78454315-d665-4fc4-9bac-de875e0b4b96 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:28.515797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:28.516395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.516695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.516957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.517629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:28.517901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:28.519526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:28.530904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 906/1831] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:28.535484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00d1a-373e-4d10-a430-b1d96250f4e1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.545081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00d1a-373e-4d10-a430-b1d96250f4e1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:28.550431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:28.551034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.551313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.551582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.552222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:28.554304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:28.585114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 926/1832] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:28 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:28.589672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.600164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836776-679a-4196-ad88-0ebd6c63c8f5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:28.631301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:28.665511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:28.666120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.666392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.666656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.667308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:28.669314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:28.696455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 907/1833] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:28 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:28.700658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.711271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03689026-2b60-4da3-bcc8-7ec5a3bc098b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:28.716004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:28.716713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.716917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.717182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.717819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:28.719403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:28.729493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 927/1834] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:28.733611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.743763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8f397-5789-47f8-a4bc-9c51a9d9a3d5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:28.748634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174e7dd-8435-4163-8868-26dc631caeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:28.749213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174e7dd-8435-4163-8868-26dc631caeeb admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.749483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174e7dd-8435-4163-8868-26dc631caeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.749758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174e7dd-8435-4163-8868-26dc631caeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.750399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174e7dd-8435-4163-8868-26dc631caeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:28.752007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174e7dd-8435-4163-8868-26dc631caeeb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:28.763255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 908/1835] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:28.767503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.777791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee473f-4069-4005-93b0-3efbcddd6c4a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:28.783355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:28.784000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.784275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.784574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] PATH_INFO: `/v3/projects/9bd04d72824642bb85f61363e50a35ed/users/452b005f93414368be630ece9d697599/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.799381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd04d72824642bb85f61363e50a35ed, user_id=452b005f93414368be630ece9d697599, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:28.802065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:28.815990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 928/1836] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:28 2022] PUT /identity/v3/projects/9bd04d72824642bb85f61363e50a35ed/users/452b005f93414368be630ece9d697599/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:28.828326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fa587-4bf8-47ea-b44f-b82f0a5553e6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:28.828698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fa587-4bf8-47ea-b44f-b82f0a5553e6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:28.828988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fa587-4bf8-47ea-b44f-b82f0a5553e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:28.859166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5e8fa587-4bf8-47ea-b44f-b82f0a5553e6 None None] MFA Rules not processed for user `452b005f93414368be630ece9d697599`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:28.860391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5e8fa587-4bf8-47ea-b44f-b82f0a5553e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:28.930349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 909/1837] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:28.935024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:28.998294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef25395-6a17-48b1-9ac9-99327f694b96 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.003692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.004292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.004596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.004864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.005488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.007463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.043478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 929/1838] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:28 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:29.047794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fa587-4bf8-47ea-b44f-b82f0a5553e6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:29.063289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fa587-4bf8-47ea-b44f-b82f0a5553e6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.124732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.125347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.125624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.125902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.126546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.128865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.153641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 910/1839] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:29 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:29.157756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:29.169033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e068-5c77-4ed6-bc41-91fd76e64ee3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.175138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.175767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.176408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.176408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.177227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.179769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.193443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 930/1840] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:29.197923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:29.209321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17835d9-8349-41db-a2c2-fe4127fa6255 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.215356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.216206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.216581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.216925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] PATH_INFO: `/v3/projects/9d0b9700cb7b4ae9a2f2c256cdc27958/users/8f29a7227da846d58811a4571c6f0725/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.233600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0b9700cb7b4ae9a2f2c256cdc27958, user_id=8f29a7227da846d58811a4571c6f0725, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.237691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.273054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 911/1841] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:29 2022] PUT /identity/v3/projects/9d0b9700cb7b4ae9a2f2c256cdc27958/users/8f29a7227da846d58811a4571c6f0725/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:29.277255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:29.288739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81413b78-baf0-4e61-9f95-72a1f1c5016c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.376700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222662e-4e1a-4279-b370-7e3899ccb6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.377714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222662e-4e1a-4279-b370-7e3899ccb6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.377966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222662e-4e1a-4279-b370-7e3899ccb6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.378250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222662e-4e1a-4279-b370-7e3899ccb6f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.378898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222662e-4e1a-4279-b370-7e3899ccb6f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.384201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222662e-4e1a-4279-b370-7e3899ccb6f5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.410512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 931/1842] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:29.411813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:29.419990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:29.425937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff7b0f-2563-4625-97e8-b2ca2b8a5d61 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.433443 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.434926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.435474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.436028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] PATH_INFO: `/v3/projects/9d0b9700cb7b4ae9a2f2c256cdc27958/users/8f29a7227da846d58811a4571c6f0725/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.440384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0b9700cb7b4ae9a2f2c256cdc27958, user_id=8f29a7227da846d58811a4571c6f0725, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.448747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.476463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 912/1843] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:29 2022] PUT /identity/v3/projects/9d0b9700cb7b4ae9a2f2c256cdc27958/users/8f29a7227da846d58811a4571c6f0725/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:29.482889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b750e2-997e-4e0e-a296-bb3389570297 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.483316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b750e2-997e-4e0e-a296-bb3389570297 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.483793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b750e2-997e-4e0e-a296-bb3389570297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.499362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:29.559320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d7b750e2-997e-4e0e-a296-bb3389570297 None None] MFA Rules not processed for user `8f29a7227da846d58811a4571c6f0725`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:29.562074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d7b750e2-997e-4e0e-a296-bb3389570297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:29.650494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 932/1844] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:29.662483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:29.664201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:29.728387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cc659-feed-4bbd-bbb3-33636bac58bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.732168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.732922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.733114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.733385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.743873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.745496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.756968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 913/1845] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:29.849637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b750e2-997e-4e0e-a296-bb3389570297 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:29.861445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b750e2-997e-4e0e-a296-bb3389570297 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:29.866123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24191ca-2ce3-4721-9300-c2c09b99782f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:29.866772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24191ca-2ce3-4721-9300-c2c09b99782f service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:29.867101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24191ca-2ce3-4721-9300-c2c09b99782f service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:29.867547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24191ca-2ce3-4721-9300-c2c09b99782f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:29.930256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24191ca-2ce3-4721-9300-c2c09b99782f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:29.933043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24191ca-2ce3-4721-9300-c2c09b99782f service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:29.942721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 933/1846] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:30.240995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:30.346599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f1f8a-fc07-4ab2-9d9f-5b9a373989d3 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:30.351874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:30.352761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:30.353026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:30.353301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] PATH_INFO: `/v3/users/452b005f93414368be630ece9d697599` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:30.354611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=452b005f93414368be630ece9d697599)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:30.356825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:30.452867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452b005f93414368be630ece9d697599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:30.472084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452b005f93414368be630ece9d697599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:30.474235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452b005f93414368be630ece9d697599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:30.521367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452b005f93414368be630ece9d697599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:30.544952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452b005f93414368be630ece9d697599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:30.570873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 914/1847] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:30 2022] DELETE /identity/v3/users/452b005f93414368be630ece9d697599 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:30.578447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:30.579559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:30.580341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:30.667414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:30.669184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:30.831014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 934/1848] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:30.849533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:30.852256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:30.871183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:30.945331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25039a-8c11-49bb-8971-c336ca5e7b09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:30.949785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:30.950438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:30.950864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:30.951143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:30.963408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:30.964966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:30.997990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 915/1849] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:31.301046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:31.303007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:31.318952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f5658-2624-463e-84d4-f5c4848e68e2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:31.330497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:31.331309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:31.331674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:31.332014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] PATH_INFO: `/v3/projects/9bd04d72824642bb85f61363e50a35ed` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:31.333543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd04d72824642bb85f61363e50a35ed)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:31.335689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:31.366376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:31.394477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd04d72824642bb85f61363e50a35ed', 'request_id': 'req-4c7611c2-e6aa-4684-be02-cfad1751c115'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.402991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd04d72824642bb85f61363e50a35ed', 'request_id': 'req-4c7611c2-e6aa-4684-be02-cfad1751c115'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.415438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd04d72824642bb85f61363e50a35ed', 'request_id': 'req-4c7611c2-e6aa-4684-be02-cfad1751c115'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.428475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] The token cache is being invalidate because project 9bd04d72824642bb85f61363e50a35ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:31.428910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.429725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.431352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 935/1850] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:31 2022] DELETE /identity/v3/projects/9bd04d72824642bb85f61363e50a35ed => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:31.436943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:31.448206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49a920-da79-4e25-8c74-95caf37d1fdd service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:31.559593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:31.560196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:31.560622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:31.561120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] PATH_INFO: `/v3/users/8f29a7227da846d58811a4571c6f0725` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:31.564310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f29a7227da846d58811a4571c6f0725)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:31.567051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:31.672383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f29a7227da846d58811a4571c6f0725'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.703539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f29a7227da846d58811a4571c6f0725'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.704481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f29a7227da846d58811a4571c6f0725'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.729127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f29a7227da846d58811a4571c6f0725'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.758367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4aae004f-ddb1-4f4f-bb3e-ef20374135d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f29a7227da846d58811a4571c6f0725'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:31.767816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 916/1851] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:31 2022] DELETE /identity/v3/users/8f29a7227da846d58811a4571c6f0725 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:32.422006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:32.424110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:32.441294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7611c2-e6aa-4684-be02-cfad1751c115 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:32.516771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:32.517371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.517799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.518858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] PATH_INFO: `/v3/projects/9d0b9700cb7b4ae9a2f2c256cdc27958` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.520103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0b9700cb7b4ae9a2f2c256cdc27958)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:32.522159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:32.601477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0b9700cb7b4ae9a2f2c256cdc27958', 'request_id': 'req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.610828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0b9700cb7b4ae9a2f2c256cdc27958', 'request_id': 'req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.617586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0b9700cb7b4ae9a2f2c256cdc27958', 'request_id': 'req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.631060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] The token cache is being invalidate because project 9d0b9700cb7b4ae9a2f2c256cdc27958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:32.631349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.632004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6a911f73-d5f0-4fc8-9786-ee13f1dd9ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.633632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 936/1852] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:32 2022] DELETE /identity/v3/projects/9d0b9700cb7b4ae9a2f2c256cdc27958 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:32.639368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befad6d-43df-479a-9bc3-90f0fb82b410 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.639867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befad6d-43df-479a-9bc3-90f0fb82b410 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.639958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befad6d-43df-479a-9bc3-90f0fb82b410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.659984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8befad6d-43df-479a-9bc3-90f0fb82b410 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:32.661159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8befad6d-43df-479a-9bc3-90f0fb82b410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:32.692566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:32.724137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 917/1853] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:32.733529 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd853523-741e-4866-be28-d147060dc771 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.733843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd853523-741e-4866-be28-d147060dc771 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.734084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd853523-741e-4866-be28-d147060dc771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.755445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fd853523-741e-4866-be28-d147060dc771 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:32.756767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fd853523-741e-4866-be28-d147060dc771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:32.767632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 937/1854] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:32.772200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befad6d-43df-479a-9bc3-90f0fb82b410 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:32.784055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befad6d-43df-479a-9bc3-90f0fb82b410 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:32.791104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:32.791724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.792079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.792258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.792961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:32.793266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:32.794912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:32.807705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 918/1855] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:32.812689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd853523-741e-4866-be28-d147060dc771 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:32.816110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:32.818113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:32.823820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd853523-741e-4866-be28-d147060dc771 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:32.830180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:32.831082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.831373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.831634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.831798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d92e8-83a3-4742-8ad0-892331c2877d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:32.832275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:32.834332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:32.837362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:32.837998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.838280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.838554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] PATH_INFO: `/v3/users/ac17c310ced14258a9254e45298d1f9d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.839737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac17c310ced14258a9254e45298d1f9d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:32.841731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:32.865581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 938/1856] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:32 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:32.869992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:32.885860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:32.890643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac17c310ced14258a9254e45298d1f9d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.900434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac17c310ced14258a9254e45298d1f9d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.901200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac17c310ced14258a9254e45298d1f9d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.908379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac17c310ced14258a9254e45298d1f9d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.921489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-643a1043-1636-4e27-9912-e2e19f0bc825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac17c310ced14258a9254e45298d1f9d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:32.927743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 919/1857] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:32 2022] DELETE /identity/v3/users/ac17c310ced14258a9254e45298d1f9d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:32.932135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.932419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.932717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.938503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b79074-3dbe-47f6-9aa4-4ea1d9599135 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:32.951925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:32.953163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:32.997081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:32.997701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:32.997983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:32.998256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:32.998895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.000893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.021836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 920/1858] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:33.024373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 939/1859] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:32 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:33.028279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:33.029628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:33.031596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:33.033253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:33.041849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd9df5-6abd-4cea-aa07-6a6c04327a95 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:33.046802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:33.047413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.047697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.047967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.048654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.050269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.060688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 921/1860] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:33.065415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:33.075190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65f364-a7d6-4eb6-9aae-5601cbdd73fa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:33.080243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:33.081083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.081363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.081647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.082302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.083980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.090220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a48584-5793-4eda-a323-36a247389257 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:33.093852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3dd43-4a30-4aa0-a43e-e5f414050e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:33.094492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3dd43-4a30-4aa0-a43e-e5f414050e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.094775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3dd43-4a30-4aa0-a43e-e5f414050e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.094903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 922/1861] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:33.095041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3dd43-4a30-4aa0-a43e-e5f414050e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.099153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:33.105948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3dd43-4a30-4aa0-a43e-e5f414050e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.107562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3dd43-4a30-4aa0-a43e-e5f414050e53 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.109969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268273e4-4c83-47e1-9fe8-dfc49429affb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:33.114876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:33.115613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.115893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.116166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] PATH_INFO: `/v3/projects/f1aa65f791484f6cb0b29da39d0e73f4/users/26c891bf39de4abfaa3a6e3aec5b90ea/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.118166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 940/1862] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:33.129469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:33.135051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1aa65f791484f6cb0b29da39d0e73f4, user_id=26c891bf39de4abfaa3a6e3aec5b90ea, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.138246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.153157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 923/1863] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:33 2022] PUT /identity/v3/projects/f1aa65f791484f6cb0b29da39d0e73f4/users/26c891bf39de4abfaa3a6e3aec5b90ea/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:33.164079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.164366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.164676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.200652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] MFA Rules not processed for user `26c891bf39de4abfaa3a6e3aec5b90ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:33.201902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:33.255713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:33.301600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 941/1864] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:33.314187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:33.316004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:33.384056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9a5b1-98c1-42c2-8a5f-b2c23ee08471 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:33.388819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:33.389555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.389840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.390200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.400663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.402146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.414116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 924/1865] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:33.446638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:33.448689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:33.549121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3568ce-79f3-444e-bc68-b85ff8613243 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:33.554192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:33.555060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.555341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.555607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] PATH_INFO: `/v3/projects/62b9c9dd59d94b3a9787e460a0f4c2fc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.560167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b9c9dd59d94b3a9787e460a0f4c2fc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.562170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.619703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b9c9dd59d94b3a9787e460a0f4c2fc', 'request_id': 'req-5e3361dd-d939-4d51-ab36-eab004ffa7f7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.638700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b9c9dd59d94b3a9787e460a0f4c2fc', 'request_id': 'req-5e3361dd-d939-4d51-ab36-eab004ffa7f7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.647915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b9c9dd59d94b3a9787e460a0f4c2fc', 'request_id': 'req-5e3361dd-d939-4d51-ab36-eab004ffa7f7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.660907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] The token cache is being invalidate because project 62b9c9dd59d94b3a9787e460a0f4c2fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:33.661194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.662963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.667988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 942/1866] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:33 2022] DELETE /identity/v3/projects/62b9c9dd59d94b3a9787e460a0f4c2fc => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:33.669232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:33.682559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cf215-e802-49e9-b9df-bb42501ee8e7 service glance] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:33.747368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:33.748324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:33.748688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:33.749015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] PATH_INFO: `/v3/users/a37b27a27e8842458009384e42b8bde9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:33.750292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37b27a27e8842458009384e42b8bde9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:33.753896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:33.841573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37b27a27e8842458009384e42b8bde9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.856325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37b27a27e8842458009384e42b8bde9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.857791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37b27a27e8842458009384e42b8bde9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.868791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37b27a27e8842458009384e42b8bde9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.885169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-df0fb4b0-3b00-43a6-a736-fcd9e2febc94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37b27a27e8842458009384e42b8bde9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:33.893965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 925/1867] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:33 2022] DELETE /identity/v3/users/a37b27a27e8842458009384e42b8bde9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:34.203309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:34.205148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:34.218942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3361dd-d939-4d51-ab36-eab004ffa7f7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:34.280404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:34.281075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.281353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.281628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] PATH_INFO: `/v3/projects/241617753e144418a0c09e3804dee2cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.282806 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] RBAC: Authorizing `identity:delete_project(project_id=241617753e144418a0c09e3804dee2cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:34.284759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:34.307612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:34.335105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241617753e144418a0c09e3804dee2cc', 'request_id': 'req-e9d4e26a-def4-428c-9df3-6d0a89dd63de'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:34.344192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241617753e144418a0c09e3804dee2cc', 'request_id': 'req-e9d4e26a-def4-428c-9df3-6d0a89dd63de'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:34.351317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241617753e144418a0c09e3804dee2cc', 'request_id': 'req-e9d4e26a-def4-428c-9df3-6d0a89dd63de'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:34.366790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] The token cache is being invalidate because project 241617753e144418a0c09e3804dee2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:34.367083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:34.367755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9d4e26a-def4-428c-9df3-6d0a89dd63de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:34.369387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 943/1868] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:34 2022] DELETE /identity/v3/projects/241617753e144418a0c09e3804dee2cc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:34.375621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569103f6-7ce5-4fa5-8063-fec77284ae43 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.375900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569103f6-7ce5-4fa5-8063-fec77284ae43 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.376170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569103f6-7ce5-4fa5-8063-fec77284ae43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.398358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-569103f6-7ce5-4fa5-8063-fec77284ae43 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:34.399580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-569103f6-7ce5-4fa5-8063-fec77284ae43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:34.405566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:34.465460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 926/1869] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:34.474394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1918f86-37c8-4f60-a5b7-925ed7765c50 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.474894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1918f86-37c8-4f60-a5b7-925ed7765c50 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.474982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1918f86-37c8-4f60-a5b7-925ed7765c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.495599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c1918f86-37c8-4f60-a5b7-925ed7765c50 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:34.496854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c1918f86-37c8-4f60-a5b7-925ed7765c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:34.505793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 944/1870] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:34.510653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569103f6-7ce5-4fa5-8063-fec77284ae43 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:34.522654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569103f6-7ce5-4fa5-8063-fec77284ae43 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:34.527610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:34.528238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.528653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.528920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.529588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:34.529852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:34.531890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:34.543909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 927/1871] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:34.550254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1918f86-37c8-4f60-a5b7-925ed7765c50 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:34.557884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1918f86-37c8-4f60-a5b7-925ed7765c50 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:34.563644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:34.564347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.564756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.565082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.565781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:34.567748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:34.596833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 945/1872] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:34 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:34.601298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:34.612586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82c072-83b4-4f58-aa17-36d29da226f8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:34.674156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:34.674836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.675044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.675312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.675950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:34.678075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:34.705281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 928/1873] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:34 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:34.706582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:34.717787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ceb49-9711-4e01-bdc6-6f8e1b002f78 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:34.724498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:34.725113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.725410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.726180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.726752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:34.727940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:34.744914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 946/1874] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:34.746306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:34.780889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4ef88-ff70-4668-8e08-57d5177bf7b1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:34.786047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2989c5b-93ba-49cf-b0f2-2be1700e8ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:34.786708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2989c5b-93ba-49cf-b0f2-2be1700e8ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.787131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2989c5b-93ba-49cf-b0f2-2be1700e8ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.787481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2989c5b-93ba-49cf-b0f2-2be1700e8ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.788215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2989c5b-93ba-49cf-b0f2-2be1700e8ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:34.789934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2989c5b-93ba-49cf-b0f2-2be1700e8ae4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:34.805270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 929/1875] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:34.810142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:34.821596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3ac37-ca21-472f-a7c8-05387a9dafe3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:34.843554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:34.844477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.845315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.846325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] PATH_INFO: `/v3/projects/1db7287dd05d4d1ca3d2fef03fca00c9/users/598b9ea9c4af457eb5093d12587254ea/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.865677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1db7287dd05d4d1ca3d2fef03fca00c9, user_id=598b9ea9c4af457eb5093d12587254ea, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:34.867618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:34.887401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 947/1876] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:34 2022] PUT /identity/v3/projects/1db7287dd05d4d1ca3d2fef03fca00c9/users/598b9ea9c4af457eb5093d12587254ea/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:34.898837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a6c3-2a31-4b52-bebb-cbcea29a31ce None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:34.899118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a6c3-2a31-4b52-bebb-cbcea29a31ce None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:34.899388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a6c3-2a31-4b52-bebb-cbcea29a31ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:34.938680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-38d0a6c3-2a31-4b52-bebb-cbcea29a31ce None None] MFA Rules not processed for user `598b9ea9c4af457eb5093d12587254ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:34.940302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-38d0a6c3-2a31-4b52-bebb-cbcea29a31ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:35.059626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 930/1877] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:35.068333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:35.070109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:35.171328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f6e99-061d-4b73-8970-a6bdfa7e41a4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:35.176238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfe97e-06c0-4c10-98e1-1ee22149d829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:35.177285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfe97e-06c0-4c10-98e1-1ee22149d829 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:35.177854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfe97e-06c0-4c10-98e1-1ee22149d829 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:35.178299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfe97e-06c0-4c10-98e1-1ee22149d829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:35.190406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfe97e-06c0-4c10-98e1-1ee22149d829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:35.192158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfe97e-06c0-4c10-98e1-1ee22149d829 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:35.203110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 948/1878] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:35.224604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a6c3-2a31-4b52-bebb-cbcea29a31ce None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:35.234479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:35.296964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a6c3-2a31-4b52-bebb-cbcea29a31ce None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:35.301680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:35.302338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:35.302669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:35.302943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] PATH_INFO: `/v3/users/598b9ea9c4af457eb5093d12587254ea` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:35.304123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=598b9ea9c4af457eb5093d12587254ea)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:35.306098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:35.358724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598b9ea9c4af457eb5093d12587254ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:35.369652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598b9ea9c4af457eb5093d12587254ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:35.370385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598b9ea9c4af457eb5093d12587254ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:35.381903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598b9ea9c4af457eb5093d12587254ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:35.400873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598b9ea9c4af457eb5093d12587254ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:35.407622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 931/1879] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:35 2022] DELETE /identity/v3/users/598b9ea9c4af457eb5093d12587254ea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:35.415634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:35.415634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:35.415634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:35.437446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:35.438729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:35.485461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:35.509973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 949/1880] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:35.521148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:35.524106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:35.589381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25d779-9a63-46f0-ae26-804c2c03e0a2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:35.595542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbcb55-dba7-4124-8352-d9c577b8708f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:35.596470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbcb55-dba7-4124-8352-d9c577b8708f service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:35.596600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbcb55-dba7-4124-8352-d9c577b8708f service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:35.596821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbcb55-dba7-4124-8352-d9c577b8708f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:35.609859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbcb55-dba7-4124-8352-d9c577b8708f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:35.611792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbcb55-dba7-4124-8352-d9c577b8708f service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:35.624159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 932/1881] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:35.941585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:35.943354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:35.973225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51899b74-ec95-4fbd-97c7-7246a30fc480 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:35.984468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:35.986134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:35.986355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:35.986628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] PATH_INFO: `/v3/projects/1db7287dd05d4d1ca3d2fef03fca00c9` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:35.988885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1db7287dd05d4d1ca3d2fef03fca00c9)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:35.991285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.082078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1db7287dd05d4d1ca3d2fef03fca00c9', 'request_id': 'req-11bad25b-672a-4f0f-8dc5-bc2010aea420'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:36.091355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1db7287dd05d4d1ca3d2fef03fca00c9', 'request_id': 'req-11bad25b-672a-4f0f-8dc5-bc2010aea420'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:36.098132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1db7287dd05d4d1ca3d2fef03fca00c9', 'request_id': 'req-11bad25b-672a-4f0f-8dc5-bc2010aea420'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:36.110987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] The token cache is being invalidate because project 1db7287dd05d4d1ca3d2fef03fca00c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:36.111284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:36.112211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-11bad25b-672a-4f0f-8dc5-bc2010aea420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:36.113990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 950/1882] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:35 2022] DELETE /identity/v3/projects/1db7287dd05d4d1ca3d2fef03fca00c9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:36.119281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f4aac-9c6a-444a-82c2-03a60cead01e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.119800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f4aac-9c6a-444a-82c2-03a60cead01e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.119800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f4aac-9c6a-444a-82c2-03a60cead01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.152201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8a7f4aac-9c6a-444a-82c2-03a60cead01e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:36.153529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8a7f4aac-9c6a-444a-82c2-03a60cead01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:36.186163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:36.270477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 933/1883] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:36.279372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382d2b-fbc2-46b8-a958-42c4b84f118d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.279708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382d2b-fbc2-46b8-a958-42c4b84f118d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.279940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382d2b-fbc2-46b8-a958-42c4b84f118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.323577 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ee382d2b-fbc2-46b8-a958-42c4b84f118d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:36.325454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ee382d2b-fbc2-46b8-a958-42c4b84f118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:36.334980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 951/1884] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:36.339982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f4aac-9c6a-444a-82c2-03a60cead01e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.350854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f4aac-9c6a-444a-82c2-03a60cead01e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.356968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:36.358627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.359065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.359476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.360857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:36.361332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:36.363731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.376866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 934/1885] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:36.381800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382d2b-fbc2-46b8-a958-42c4b84f118d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.391206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382d2b-fbc2-46b8-a958-42c4b84f118d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.397124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:36.397875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.398214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.398555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.399274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:36.401291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.450043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 952/1886] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:36 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:36.454892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.471428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef4f61-8f8e-48ad-887f-44f0bc4bb821 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.534802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:36.535410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.535686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.535955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.536632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:36.538640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.565632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 935/1887] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:36 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:36.567100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.579245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339393a-1e97-4d2e-ac8a-7cc39b301d98 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.585549 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:36.586164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.586440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.586709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.587356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:36.589218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.600255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 953/1888] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:36.604555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.614843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a9784-1931-41af-9ef4-75c916939810 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.619731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375b7ac-7e89-4b26-8aab-3481a97da5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:36.620361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375b7ac-7e89-4b26-8aab-3481a97da5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.620664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375b7ac-7e89-4b26-8aab-3481a97da5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.620931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375b7ac-7e89-4b26-8aab-3481a97da5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.621577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375b7ac-7e89-4b26-8aab-3481a97da5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:36.623228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375b7ac-7e89-4b26-8aab-3481a97da5ac admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.634431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 936/1889] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:36.638893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.650539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c9138-06f1-4e55-be31-67d8dbe247cc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.656068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:36.656786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.657062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.657315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] PATH_INFO: `/v3/projects/bb3eb077e19e40f2b7eb1f0f29671c6a/users/04a7c7f0601f4a5286d8c8e4e011cae0/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.673792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3eb077e19e40f2b7eb1f0f29671c6a, user_id=04a7c7f0601f4a5286d8c8e4e011cae0, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:36.676472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.691891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 954/1890] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:36 2022] PUT /identity/v3/projects/bb3eb077e19e40f2b7eb1f0f29671c6a/users/04a7c7f0601f4a5286d8c8e4e011cae0/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:36.702779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9ba0-d52b-4a02-9989-3617517a51e3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.703039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9ba0-d52b-4a02-9989-3617517a51e3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.703347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f9ba0-d52b-4a02-9989-3617517a51e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.735836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f69f9ba0-d52b-4a02-9989-3617517a51e3 None None] MFA Rules not processed for user `04a7c7f0601f4a5286d8c8e4e011cae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:36.737101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f69f9ba0-d52b-4a02-9989-3617517a51e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:36.805676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 937/1891] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:36.810467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.887485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e836f6-7c40-47ec-be0c-932f4ab98a53 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.892716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:36.893335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:36.893615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:36.893888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:36.894532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:36.896563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:36.902372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:36.931154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 955/1892] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:36 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:36.935808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f9ba0-d52b-4a02-9989-3617517a51e3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:36.948005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f9ba0-d52b-4a02-9989-3617517a51e3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:36.969163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:37.013628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.014250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.014521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.014790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.015431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.017434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.041033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 938/1893] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:36 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:37.045161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.056438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93186ba3-d574-47e1-ab52-33dbc04684ab admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.062554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.063168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.063434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.063710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.064411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.066090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.078024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 956/1894] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:37.081808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.098325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fc309-d230-42f5-b205-5dbcd3a90577 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.103296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87be200-83c1-45ea-8c1c-cf6e155c35e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.103976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be200-83c1-45ea-8c1c-cf6e155c35e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.104186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be200-83c1-45ea-8c1c-cf6e155c35e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.104492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87be200-83c1-45ea-8c1c-cf6e155c35e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.105188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87be200-83c1-45ea-8c1c-cf6e155c35e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.106764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87be200-83c1-45ea-8c1c-cf6e155c35e5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.117270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 939/1895] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:37.121909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.133648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659b640-0296-4ef6-8ac6-31b1e1783914 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.141269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.142194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.142473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.142745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] PATH_INFO: `/v3/projects/94e1a476ca154f8aa166b7d51cca89d8/users/a79eb99ddd39400980d5efe92d9d9916/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.163013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e1a476ca154f8aa166b7d51cca89d8, user_id=a79eb99ddd39400980d5efe92d9d9916, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.165928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.182202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 957/1896] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:37 2022] PUT /identity/v3/projects/94e1a476ca154f8aa166b7d51cca89d8/users/a79eb99ddd39400980d5efe92d9d9916/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:37.195125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00151d83-ca75-47a4-8810-f0e0bbcca708 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.196097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00151d83-ca75-47a4-8810-f0e0bbcca708 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.196097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00151d83-ca75-47a4-8810-f0e0bbcca708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.226661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-00151d83-ca75-47a4-8810-f0e0bbcca708 None None] MFA Rules not processed for user `a79eb99ddd39400980d5efe92d9d9916`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:37.227986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-00151d83-ca75-47a4-8810-f0e0bbcca708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:37.332489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 940/1897] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:37.337049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.406916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f919c6e-f72f-4b2e-8ac0-25cf19fac714 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.412150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.412864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.413140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.413451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.414090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.416020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.444920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 958/1898] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:37 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:37.450611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00151d83-ca75-47a4-8810-f0e0bbcca708 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.462582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00151d83-ca75-47a4-8810-f0e0bbcca708 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.525870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.526499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.526770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.527036 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.527680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.529789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.563016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 941/1899] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:37 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:37.567246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.588051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b3198-6273-4494-a6cb-3578e99b9fdc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.602826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.603461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.603740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.604064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.604763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.607290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.621499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 959/1900] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:37.625775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.639581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecf6b-7443-43b5-92e2-a8552490c566 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.645033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.645788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.646131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.646593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] PATH_INFO: `/v3/projects/a6a5706fafc14df1b1bdcea9a37e5b29/users/7ecca1c0c1a74e639ff7e684e66701cb/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.668566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a5706fafc14df1b1bdcea9a37e5b29, user_id=7ecca1c0c1a74e639ff7e684e66701cb, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.671915 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.695871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:37.702432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 942/1901] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:37 2022] PUT /identity/v3/projects/a6a5706fafc14df1b1bdcea9a37e5b29/users/7ecca1c0c1a74e639ff7e684e66701cb/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:37.707885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.725663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa033543-38bb-4046-8f39-476a2dd3b330 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.848694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b532fa-73d1-4f21-8f04-94ac0e9f81aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.849364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b532fa-73d1-4f21-8f04-94ac0e9f81aa admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.849679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b532fa-73d1-4f21-8f04-94ac0e9f81aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.849952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b532fa-73d1-4f21-8f04-94ac0e9f81aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.850609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b532fa-73d1-4f21-8f04-94ac0e9f81aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.852237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b532fa-73d1-4f21-8f04-94ac0e9f81aa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.877101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 960/1902] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:37.881842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:37.895177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95976b-9ba2-412e-bb4b-8d977ff34858 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:37.904177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:37.904924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.905181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.905444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] PATH_INFO: `/v3/projects/a6a5706fafc14df1b1bdcea9a37e5b29/users/7ecca1c0c1a74e639ff7e684e66701cb/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.907892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a5706fafc14df1b1bdcea9a37e5b29, user_id=7ecca1c0c1a74e639ff7e684e66701cb, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:37.910926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:37.922391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 943/1903] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:37 2022] PUT /identity/v3/projects/a6a5706fafc14df1b1bdcea9a37e5b29/users/7ecca1c0c1a74e639ff7e684e66701cb/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:37.939251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279649dd-1b64-483f-9a43-0223703d81b0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:37.939696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279649dd-1b64-483f-9a43-0223703d81b0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:37.940064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279649dd-1b64-483f-9a43-0223703d81b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:37.976897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-279649dd-1b64-483f-9a43-0223703d81b0 None None] MFA Rules not processed for user `7ecca1c0c1a74e639ff7e684e66701cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:37.978147 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-279649dd-1b64-483f-9a43-0223703d81b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:37.987730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:38.059471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 961/1904] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:38.072248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:38.074181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:38.138096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f94d6-370e-431b-b4a8-908ddba387f9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:38.142234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:38.142898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:38.143173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:38.143439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:38.207730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:38.209325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:38.219585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 944/1905] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:41.196773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279649dd-1b64-483f-9a43-0223703d81b0 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:41.210080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279649dd-1b64-483f-9a43-0223703d81b0 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:41.215541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:41.216198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:41.216588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:41.216921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:41.286417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:41.286417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:41.293562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 962/1906] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:42.731854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:42.733542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:42.747739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77704152-ed2a-45ea-b589-95f6d562fb46 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:42.752281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf6b55-3070-47c8-a4fd-18ac6f1e2a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:42.753373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6b55-3070-47c8-a4fd-18ac6f1e2a6c service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:42.753719 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6b55-3070-47c8-a4fd-18ac6f1e2a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:42.753995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf6b55-3070-47c8-a4fd-18ac6f1e2a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:42.764714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6b55-3070-47c8-a4fd-18ac6f1e2a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:42.767427 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf6b55-3070-47c8-a4fd-18ac6f1e2a6c service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:42.779504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 945/1907] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:46.145856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:46.148999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:46.233104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255e379-df5f-45a0-8e85-b0f12f94e749 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:46.240838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:46.241979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:46.242316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:46.242592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] PATH_INFO: `/v3/users/5ba7e99def8f4744946e7b1697f55a44` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:46.243783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba7e99def8f4744946e7b1697f55a44)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:46.245827 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:46.301930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba7e99def8f4744946e7b1697f55a44'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:46.304717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:46.315384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba7e99def8f4744946e7b1697f55a44'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:46.316282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba7e99def8f4744946e7b1697f55a44'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:46.325129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba7e99def8f4744946e7b1697f55a44'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:46.344796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba7e99def8f4744946e7b1697f55a44'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:46.353236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 963/1908] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:46 2022] DELETE /identity/v3/users/5ba7e99def8f4744946e7b1697f55a44 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:46.358120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:46.358686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:46.358991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:46.387585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:46.389062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:46.549794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 946/1909] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:46.565651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:46.571053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:46.673004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181a9f4-ba81-4a69-b364-8d555782407a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:46.685281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:46.686584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:46.687100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:46.687695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:46.715396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:46.715396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:46.725211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 964/1910] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:47.102426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:47.104404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:47.120734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49845d5-7469-4471-b010-e024a4c692e9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:47.132982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:47.133892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:47.134551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:47.134944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] PATH_INFO: `/v3/projects/e9c4a325015941f8b09c0e6c3ba50d63` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:47.136322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c4a325015941f8b09c0e6c3ba50d63)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:47.138319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:47.166522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:47.207978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c4a325015941f8b09c0e6c3ba50d63', 'request_id': 'req-18c69091-66a3-4a76-b881-08ea53486758'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.218191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c4a325015941f8b09c0e6c3ba50d63', 'request_id': 'req-18c69091-66a3-4a76-b881-08ea53486758'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.225410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c4a325015941f8b09c0e6c3ba50d63', 'request_id': 'req-18c69091-66a3-4a76-b881-08ea53486758'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.237873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] The token cache is being invalidate because project e9c4a325015941f8b09c0e6c3ba50d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:47.238302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.239273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.241811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 947/1911] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:47 2022] DELETE /identity/v3/projects/e9c4a325015941f8b09c0e6c3ba50d63 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:47.245693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:47.258074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdea92c-a644-477c-97a2-ee6834d79b56 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:47.323774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:47.324550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:47.324902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:47.325244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] PATH_INFO: `/v3/users/abd7d5312bab49d28f84e2991e31f351` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:47.326454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd7d5312bab49d28f84e2991e31f351)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:47.328484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:47.375880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd7d5312bab49d28f84e2991e31f351'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.387012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd7d5312bab49d28f84e2991e31f351'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.387804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd7d5312bab49d28f84e2991e31f351'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.396322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd7d5312bab49d28f84e2991e31f351'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.401959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:47.410779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a4873c53-6bac-42ad-b570-3e50b0d6037c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd7d5312bab49d28f84e2991e31f351'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.417670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 965/1912] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:47 2022] DELETE /identity/v3/users/abd7d5312bab49d28f84e2991e31f351 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:47.753439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:47.755131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:47.771080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c69091-66a3-4a76-b881-08ea53486758 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:47.831569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:47.832216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:47.832526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:47.832801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] PATH_INFO: `/v3/projects/5ec56dea8111409f8e51794a74f81c7f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:47.833933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec56dea8111409f8e51794a74f81c7f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:47.835995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:47.893493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec56dea8111409f8e51794a74f81c7f', 'request_id': 'req-42669b4d-b270-47a8-b292-d9cd035c0f36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.909934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec56dea8111409f8e51794a74f81c7f', 'request_id': 'req-42669b4d-b270-47a8-b292-d9cd035c0f36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.917081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec56dea8111409f8e51794a74f81c7f', 'request_id': 'req-42669b4d-b270-47a8-b292-d9cd035c0f36'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.927384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] The token cache is being invalidate because project 5ec56dea8111409f8e51794a74f81c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:47.927647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.928361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42669b4d-b270-47a8-b292-d9cd035c0f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:47.930057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 948/1913] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:47 2022] DELETE /identity/v3/projects/5ec56dea8111409f8e51794a74f81c7f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:47.935811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1abfe-0290-4cb3-8dc7-7673b7bcb3d3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:47.936121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1abfe-0290-4cb3-8dc7-7673b7bcb3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:47.936585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1abfe-0290-4cb3-8dc7-7673b7bcb3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:47.959360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-afd1abfe-0290-4cb3-8dc7-7673b7bcb3d3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:47.960711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-afd1abfe-0290-4cb3-8dc7-7673b7bcb3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:48.029781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 966/1914] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:48.042179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c3d3-fd5f-44ea-9a67-53ed9e59f182 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.042652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c3d3-fd5f-44ea-9a67-53ed9e59f182 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.042972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23c3d3-fd5f-44ea-9a67-53ed9e59f182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.066596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ca23c3d3-fd5f-44ea-9a67-53ed9e59f182 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:48.067812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ca23c3d3-fd5f-44ea-9a67-53ed9e59f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:48.076365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 949/1915] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:48.080703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1abfe-0290-4cb3-8dc7-7673b7bcb3d3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.090829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1abfe-0290-4cb3-8dc7-7673b7bcb3d3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.095765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.096378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.096681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.096951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.097671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:48.097926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.099532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.111321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 967/1916] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:48.115924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23c3d3-fd5f-44ea-9a67-53ed9e59f182 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.125608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23c3d3-fd5f-44ea-9a67-53ed9e59f182 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.130779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.131392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.131678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.131948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.132633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.134579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.168176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 950/1917] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:48 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:48.169574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.179921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b1d6d-a439-40d5-ab2f-d813969d00b4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.238574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.239187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.239465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.239757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.240375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.242336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.265135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 968/1918] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:48 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:48.269377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.279736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cb29d-8e5b-4cc5-b8c4-0f48f964e3e8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.285130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.285727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.286001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.286301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.286947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.288663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.294359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:48.301246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 951/1919] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:48.305189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.315625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91f444-a18c-4cbd-8f25-70c4a91e7209 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.320315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b70b5-4370-472b-84dd-9138f6c1079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.320958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b70b5-4370-472b-84dd-9138f6c1079a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.321236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b70b5-4370-472b-84dd-9138f6c1079a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.321508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b70b5-4370-472b-84dd-9138f6c1079a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.322158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b70b5-4370-472b-84dd-9138f6c1079a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.323802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b70b5-4370-472b-84dd-9138f6c1079a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.336622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 969/1920] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:48.341338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.352611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9521cf-6938-451f-9dc4-5d1cc8ad2114 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.357403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.358905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.358905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.358905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] PATH_INFO: `/v3/projects/8061f9428159473badf8685e6541bb51/users/446c4fb3025f449a915b2845e906515d/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.380691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8061f9428159473badf8685e6541bb51, user_id=446c4fb3025f449a915b2845e906515d, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.383703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.398494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 952/1921] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:48 2022] PUT /identity/v3/projects/8061f9428159473badf8685e6541bb51/users/446c4fb3025f449a915b2845e906515d/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:48.410087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365755dc-1e2a-446c-9941-375e149fcc54 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.410382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365755dc-1e2a-446c-9941-375e149fcc54 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.410654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365755dc-1e2a-446c-9941-375e149fcc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.441794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-365755dc-1e2a-446c-9941-375e149fcc54 None None] MFA Rules not processed for user `446c4fb3025f449a915b2845e906515d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:48.443035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-365755dc-1e2a-446c-9941-375e149fcc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:48.506154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:48.532192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 970/1922] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:48.538767 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.605500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250b6a7-56e2-4b29-a53e-1ae76a9e2d54 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.610543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.611144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.611425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.611696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.612340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.614345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.648295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 953/1923] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:48 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:48.652730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365755dc-1e2a-446c-9941-375e149fcc54 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.664567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365755dc-1e2a-446c-9941-375e149fcc54 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.724884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.725611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.726026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.726283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.727084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.734027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.756418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 971/1924] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:48 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:48.758159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.770682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c6c06-73ec-47e3-8ddd-3be8873745cf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.776173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.776843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.777124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.777395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.778053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.779693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.792442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 954/1925] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:48.799943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.810450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da3b8c-58a9-4922-9fb7-7cbea7eaabc5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.816080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.816824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.817102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.817377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] PATH_INFO: `/v3/projects/a09b578e962f43bd81fcf7a39d731b1c/users/15469ec81237459dbcdb29e608c1a792/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.836392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09b578e962f43bd81fcf7a39d731b1c, user_id=15469ec81237459dbcdb29e608c1a792, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.839249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.864736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 972/1926] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:48 2022] PUT /identity/v3/projects/a09b578e962f43bd81fcf7a39d731b1c/users/15469ec81237459dbcdb29e608c1a792/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:48.864873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.871289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef59b9-12f4-4781-a36d-a0aca1a81944 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.933674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf507800-23e4-499f-8835-b1cb61d6abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.934321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf507800-23e4-499f-8835-b1cb61d6abf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.934714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf507800-23e4-499f-8835-b1cb61d6abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.934990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf507800-23e4-499f-8835-b1cb61d6abf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.935666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf507800-23e4-499f-8835-b1cb61d6abf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.937344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf507800-23e4-499f-8835-b1cb61d6abf2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.951384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 955/1927] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:48.952885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:48.965441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972775b-5516-46ed-a151-2b79d8563b3f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:48.970751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:48.971437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:48.971873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:48.972150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] PATH_INFO: `/v3/projects/a09b578e962f43bd81fcf7a39d731b1c/users/15469ec81237459dbcdb29e608c1a792/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:48.974362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09b578e962f43bd81fcf7a39d731b1c, user_id=15469ec81237459dbcdb29e608c1a792, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:48.977172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:48.988588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 973/1928] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:48 2022] PUT /identity/v3/projects/a09b578e962f43bd81fcf7a39d731b1c/users/15469ec81237459dbcdb29e608c1a792/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:48.999954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb52fb-455e-4c38-b24b-33d57fefa330 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:49.000256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb52fb-455e-4c38-b24b-33d57fefa330 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:49.000535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb52fb-455e-4c38-b24b-33d57fefa330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:49.039899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-61fb52fb-455e-4c38-b24b-33d57fefa330 None None] MFA Rules not processed for user `15469ec81237459dbcdb29e608c1a792`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:49.039899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-61fb52fb-455e-4c38-b24b-33d57fefa330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:49.072053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:49.110546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 956/1929] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:49.121709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:49.123627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:49.195903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046d188-ca52-4237-a57d-4cfb0abe776c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:49.200484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:49.201115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:49.201397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:49.201676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:49.211867 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:49.213573 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:49.224260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 974/1930] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:49.302238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb52fb-455e-4c38-b24b-33d57fefa330 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:49.312686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb52fb-455e-4c38-b24b-33d57fefa330 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:49.316805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca672e6-ae52-4659-8f88-3506483fc987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:49.317410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca672e6-ae52-4659-8f88-3506483fc987 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:49.317701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca672e6-ae52-4659-8f88-3506483fc987 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:49.317946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca672e6-ae52-4659-8f88-3506483fc987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:49.378366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca672e6-ae52-4659-8f88-3506483fc987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:49.379735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca672e6-ae52-4659-8f88-3506483fc987 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:49.389139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 957/1931] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:51.572170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:51.573880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:51.629294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:51.658692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f1742-fdb0-4960-aa69-fd153aadf210 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:51.663821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:51.664464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:51.664764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:51.665130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] PATH_INFO: `/v3/users/04a7c7f0601f4a5286d8c8e4e011cae0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:51.666315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a7c7f0601f4a5286d8c8e4e011cae0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:51.668383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:51.722490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a7c7f0601f4a5286d8c8e4e011cae0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:51.732108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a7c7f0601f4a5286d8c8e4e011cae0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:51.732851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a7c7f0601f4a5286d8c8e4e011cae0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:51.745473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a7c7f0601f4a5286d8c8e4e011cae0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:51.758728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a7c7f0601f4a5286d8c8e4e011cae0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:51.764768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 975/1932] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:51 2022] DELETE /identity/v3/users/04a7c7f0601f4a5286d8c8e4e011cae0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:51.769407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:51.769949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:51.770163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:51.790984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:51.792237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:51.867757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 958/1933] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:51.880254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:51.882575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:51.997302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601728ca-68c0-4cde-94fa-8c7d04622f0b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:52.004233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221b0fc-d6e4-441a-ab9f-2f74c247e62e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:52.005238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b0fc-d6e4-441a-ab9f-2f74c247e62e service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:52.005692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b0fc-d6e4-441a-ab9f-2f74c247e62e service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:52.006142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221b0fc-d6e4-441a-ab9f-2f74c247e62e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:52.031713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221b0fc-d6e4-441a-ab9f-2f74c247e62e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:52.031713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221b0fc-d6e4-441a-ab9f-2f74c247e62e service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:52.049654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 976/1934] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:52.226984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:52.228850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:52.253512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72555512-081d-4784-922f-de8d7c826bd7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:52.259420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:52.260459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:52.263123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:52.263533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] PATH_INFO: `/v3/users/446c4fb3025f449a915b2845e906515d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:52.265047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=446c4fb3025f449a915b2845e906515d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:52.267938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:52.369475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446c4fb3025f449a915b2845e906515d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:52.383896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446c4fb3025f449a915b2845e906515d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:52.386620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446c4fb3025f449a915b2845e906515d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:52.407703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446c4fb3025f449a915b2845e906515d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:52.411525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:52.456832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446c4fb3025f449a915b2845e906515d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:52.468517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 959/1935] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:52 2022] DELETE /identity/v3/users/446c4fb3025f449a915b2845e906515d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:52.474193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:52.474919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:52.475534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:52.501960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:52.514861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:52.585645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:52.601517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 977/1936] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:52.621185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:52.622779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:52.746415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2dfc8-5c59-49e5-9e27-966185c80df8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:52.756237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:52.758249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:52.758783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:52.759239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:52.775453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:52.786717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:52.803330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 960/1937] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:53.256163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:53.257716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:53.264939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:53.266946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:53.270665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c51ce4-3977-4c68-b817-d1beab4f24b1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:53.275637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:53.276352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:53.276773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:53.276947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] PATH_INFO: `/v3/projects/bb3eb077e19e40f2b7eb1f0f29671c6a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:53.278302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3eb077e19e40f2b7eb1f0f29671c6a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:53.280264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:53.283291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e129d-8602-420e-978b-e0b9050e22f1 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:53.288132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:53.288813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:53.289094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:53.289364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] PATH_INFO: `/v3/projects/8061f9428159473badf8685e6541bb51` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:53.290614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8061f9428159473badf8685e6541bb51)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:53.292783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:53.351782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8061f9428159473badf8685e6541bb51', 'request_id': 'req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.364393 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8061f9428159473badf8685e6541bb51', 'request_id': 'req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.373580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8061f9428159473badf8685e6541bb51', 'request_id': 'req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.387018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3eb077e19e40f2b7eb1f0f29671c6a', 'request_id': 'req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.388995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] The token cache is being invalidate because project 8061f9428159473badf8685e6541bb51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:53.389325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.390159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.395658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 961/1938] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:53 2022] DELETE /identity/v3/projects/8061f9428159473badf8685e6541bb51 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:53.396717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:53.398796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:53.402126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3eb077e19e40f2b7eb1f0f29671c6a', 'request_id': 'req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.416266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3eb077e19e40f2b7eb1f0f29671c6a', 'request_id': 'req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.434516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] The token cache is being invalidate because project bb3eb077e19e40f2b7eb1f0f29671c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:53.434842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.435536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.440055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 978/1939] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:53 2022] DELETE /identity/v3/projects/bb3eb077e19e40f2b7eb1f0f29671c6a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:53.444494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:53.448023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:53.493336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a47c49-86f3-4bad-8bb4-6722d79b17b3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:53.519150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:53.563113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b4dc6-d53e-47e7-ba1a-f5e34dce3bac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:53.576675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:53.577445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:53.578026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:53.578496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] PATH_INFO: `/v3/users/a79eb99ddd39400980d5efe92d9d9916` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:53.581608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79eb99ddd39400980d5efe92d9d9916)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:53.583825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:53.593179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:53.593865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:53.594139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:53.596819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] PATH_INFO: `/v3/users/15469ec81237459dbcdb29e608c1a792` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:53.598369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15469ec81237459dbcdb29e608c1a792)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:53.603272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:53.646773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79eb99ddd39400980d5efe92d9d9916'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.657333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79eb99ddd39400980d5efe92d9d9916'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.658555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79eb99ddd39400980d5efe92d9d9916'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.667838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79eb99ddd39400980d5efe92d9d9916'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.683389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15469ec81237459dbcdb29e608c1a792'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.687592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79eb99ddd39400980d5efe92d9d9916'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.695348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15469ec81237459dbcdb29e608c1a792'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.695555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 979/1940] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:53 2022] DELETE /identity/v3/users/a79eb99ddd39400980d5efe92d9d9916 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:53.699248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15469ec81237459dbcdb29e608c1a792'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.708124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15469ec81237459dbcdb29e608c1a792'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.726735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15469ec81237459dbcdb29e608c1a792'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:53.735186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 962/1941] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:53 2022] DELETE /identity/v3/users/15469ec81237459dbcdb29e608c1a792 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:54.273169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:54.275132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:54.289883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:54.292058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:54.292702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:54.412721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e7e22-3885-4e91-8127-3c8ca111a7b7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:54.413674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f635bfe-1469-4d92-be08-b21f3b44e9ad admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:54.418058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:54.418705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.419061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.419207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:54.419312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] PATH_INFO: `/v3/projects/94e1a476ca154f8aa166b7d51cca89d8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.419965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.420458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.420586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e1a476ca154f8aa166b7d51cca89d8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:54.421096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] PATH_INFO: `/v3/projects/a09b578e962f43bd81fcf7a39d731b1c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.422446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09b578e962f43bd81fcf7a39d731b1c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:54.423409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:54.424769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:54.471711 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:54.484019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e1a476ca154f8aa166b7d51cca89d8', 'request_id': 'req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.489742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09b578e962f43bd81fcf7a39d731b1c', 'request_id': 'req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.496396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e1a476ca154f8aa166b7d51cca89d8', 'request_id': 'req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.499706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09b578e962f43bd81fcf7a39d731b1c', 'request_id': 'req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.506408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e1a476ca154f8aa166b7d51cca89d8', 'request_id': 'req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.510519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09b578e962f43bd81fcf7a39d731b1c', 'request_id': 'req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.528130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] The token cache is being invalidate because project a09b578e962f43bd81fcf7a39d731b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:54.528393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.529156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1f72fed6-6b41-4b82-9f92-ba5b2f441ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.530706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 980/1942] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:54 2022] DELETE /identity/v3/projects/a09b578e962f43bd81fcf7a39d731b1c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:54.535675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b40f-9ad6-4291-89e2-38d187c7201a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.536051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b40f-9ad6-4291-89e2-38d187c7201a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.536444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b40f-9ad6-4291-89e2-38d187c7201a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.537801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] The token cache is being invalidate because project 94e1a476ca154f8aa166b7d51cca89d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:54.538102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.539368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.544345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 963/1943] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:54 2022] DELETE /identity/v3/projects/94e1a476ca154f8aa166b7d51cca89d8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:54.549485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:54.552701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:54.566186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8c85b40f-9ad6-4291-89e2-38d187c7201a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:54.567568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8c85b40f-9ad6-4291-89e2-38d187c7201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:54.638094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc2fc4-6d3d-42ce-9503-133d25bd36a4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:54.639995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 981/1944] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:54.647774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66771d40-930f-417e-a894-46a818847326 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.648351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66771d40-930f-417e-a894-46a818847326 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.648351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66771d40-930f-417e-a894-46a818847326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.650731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:54.651607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.652048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.652496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] PATH_INFO: `/v3/users/7ecca1c0c1a74e639ff7e684e66701cb` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.653719 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecca1c0c1a74e639ff7e684e66701cb)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:54.656022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:54.673184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-66771d40-930f-417e-a894-46a818847326 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:54.675238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-66771d40-930f-417e-a894-46a818847326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:54.686088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 982/1945] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:54.691330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66771d40-930f-417e-a894-46a818847326 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:54.701488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66771d40-930f-417e-a894-46a818847326 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:54.706815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:54.707436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.707741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.708173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.708901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:54.709201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:54.710914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:54.712208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecca1c0c1a74e639ff7e684e66701cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.721608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecca1c0c1a74e639ff7e684e66701cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.722388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecca1c0c1a74e639ff7e684e66701cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.727161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 983/1946] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:54.729244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:54.730983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:54.735414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecca1c0c1a74e639ff7e684e66701cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.746894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6680547-7566-49f5-9c6e-13555841b1c3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:54.751265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecca1c0c1a74e639ff7e684e66701cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:54.753200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:54.753831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.754101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.754367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.754996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:54.757136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:54.759785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 964/1947] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:54 2022] DELETE /identity/v3/users/7ecca1c0c1a74e639ff7e684e66701cb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:54.770414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:54.793640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 984/1948] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:54 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:54.798223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:54.856218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:54.875825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763cbaf4-432b-4d1e-a267-5b848ba0307a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:54.938369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:54.939159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.939468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.939813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.940656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:54.942732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:54.966461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 965/1949] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:54 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:54.970671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:54.981178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8de6e-b0c7-4db9-9f99-d55c356a4e20 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:54.985933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:54.986565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:54.986839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:54.987122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:54.987768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:54.989457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.000010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 985/1950] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:55.004489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.014839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae9b51-65bf-46f7-b382-ecd487d395b9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.019716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.020334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.020637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.020904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.021553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.023232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.036480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 966/1951] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:55.039055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.050680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb911-3cbd-4bf4-906f-57428e0f2ae6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.055690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1209e-b83c-4cef-b5f0-b5844b2e2443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.056487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1209e-b83c-4cef-b5f0-b5844b2e2443 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.056771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1209e-b83c-4cef-b5f0-b5844b2e2443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.057048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1209e-b83c-4cef-b5f0-b5844b2e2443 admin admin] PATH_INFO: `/v3/projects/0c1f844a3b0e4db0a20b30ab171cfdcd/users/aee14af6ca12448e8ff7fd2491573911/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.058422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.061236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.076907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1209e-b83c-4cef-b5f0-b5844b2e2443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1f844a3b0e4db0a20b30ab171cfdcd, user_id=aee14af6ca12448e8ff7fd2491573911, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.078114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63bf7-fb83-4ec2-b254-698c490d59ca admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.083757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.084474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.084754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.085169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] PATH_INFO: `/v3/projects/a6a5706fafc14df1b1bdcea9a37e5b29` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.086420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a5706fafc14df1b1bdcea9a37e5b29)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.088138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1209e-b83c-4cef-b5f0-b5844b2e2443 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.088248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.104350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 986/1952] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:55 2022] PUT /identity/v3/projects/0c1f844a3b0e4db0a20b30ab171cfdcd/users/aee14af6ca12448e8ff7fd2491573911/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:55.116329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.116550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.116820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.151661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] MFA Rules not processed for user `aee14af6ca12448e8ff7fd2491573911`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:55.153284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.158666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a5706fafc14df1b1bdcea9a37e5b29', 'request_id': 'req-6985566a-84fe-4650-a806-09df30ac5cd8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.171131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a5706fafc14df1b1bdcea9a37e5b29', 'request_id': 'req-6985566a-84fe-4650-a806-09df30ac5cd8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.187809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a5706fafc14df1b1bdcea9a37e5b29', 'request_id': 'req-6985566a-84fe-4650-a806-09df30ac5cd8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.208367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] The token cache is being invalidate because project a6a5706fafc14df1b1bdcea9a37e5b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:55.208845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.209742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6985566a-84fe-4650-a806-09df30ac5cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.212483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 967/1953] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:55 2022] DELETE /identity/v3/projects/a6a5706fafc14df1b1bdcea9a37e5b29 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:55.217933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.218451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.218753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.239547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:55.239886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 987/1954] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:55.241332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.244401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.246342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.319023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:55.320989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af33f1-eef2-4072-bc65-6e19e685bbd3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.326170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a0fc2-dadf-4897-a55c-4e5b32304a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.326739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a0fc2-dadf-4897-a55c-4e5b32304a02 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.327020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a0fc2-dadf-4897-a55c-4e5b32304a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.327293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a0fc2-dadf-4897-a55c-4e5b32304a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.328043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a0fc2-dadf-4897-a55c-4e5b32304a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.330055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a0fc2-dadf-4897-a55c-4e5b32304a02 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.344602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 968/1955] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:55.345704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.347323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.361063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e5aa8-3e3a-48a8-adcf-ed67ff5949ab None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.361616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 988/1956] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:55.363066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739c091-2fc3-4668-afb6-1b1f87f66776 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.363363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739c091-2fc3-4668-afb6-1b1f87f66776 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.363641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739c091-2fc3-4668-afb6-1b1f87f66776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.384930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-b739c091-2fc3-4668-afb6-1b1f87f66776 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:55.386263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b739c091-2fc3-4668-afb6-1b1f87f66776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.418443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:55.436885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.437691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.437970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.438400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] PATH_INFO: `/v3/users/05905557e1734182bfed315366345c77` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.439559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05905557e1734182bfed315366345c77)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.441583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.452773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 989/1957] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:55.456935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739c091-2fc3-4668-afb6-1b1f87f66776 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.466086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739c091-2fc3-4668-afb6-1b1f87f66776 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.471421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.472022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.472296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.473045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.473237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.475153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.491045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05905557e1734182bfed315366345c77'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.497603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 990/1958] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/users => generated 299 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:55.498430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.499108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05905557e1734182bfed315366345c77'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.499790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05905557e1734182bfed315366345c77'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.507910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05905557e1734182bfed315366345c77'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.508799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126d879-ebae-4aeb-af47-f84de028f7e1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.513483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c736e-e3a0-4d6c-bb14-b6e3b24cbf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.514121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c736e-e3a0-4d6c-bb14-b6e3b24cbf88 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.514424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c736e-e3a0-4d6c-bb14-b6e3b24cbf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.514695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c736e-e3a0-4d6c-bb14-b6e3b24cbf88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.515344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c736e-e3a0-4d6c-bb14-b6e3b24cbf88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:55.515645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c736e-e3a0-4d6c-bb14-b6e3b24cbf88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.517311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c736e-e3a0-4d6c-bb14-b6e3b24cbf88 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.520568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05905557e1734182bfed315366345c77'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:55.527726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 969/1959] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:55 2022] DELETE /identity/v3/users/05905557e1734182bfed315366345c77 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:55.529285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.530138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 991/1960] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:55.531135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.532030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.532317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.532618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.552210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:55.553512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.602454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200975d2-25ac-4ad7-98fc-47c641a2ee59 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.608104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.608762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.609059 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.609327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.609984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.611607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.615053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 992/1961] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:55.615946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.619421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.634174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 970/1962] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:55.634174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.634516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b3f83-1ac1-4b93-8f40-b2e29d99ec9a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.634648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:55.636919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.637557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.637833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.638097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.638731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.640749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.644513 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:55.676921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 993/1963] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:55.676921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.713108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b188a4-114f-4c03-8892-3bb187dbc03c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.749087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccf509-af4d-49b6-ab54-9ed3e24968d6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.754639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.755506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.755781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.756051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] PATH_INFO: `/v3/projects/984935c4f13b4faa8cc18f56535e31a0/users/aff1d3657c8843649c6496622d215836/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.765777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.766398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.766677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.766956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.770650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984935c4f13b4faa8cc18f56535e31a0, user_id=aff1d3657c8843649c6496622d215836, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.773432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.776811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.778630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.787765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 994/1964] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:55 2022] PUT /identity/v3/projects/984935c4f13b4faa8cc18f56535e31a0/users/aff1d3657c8843649c6496622d215836/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:55.788701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.789903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 971/1965] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:55.791665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.795812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:55.802271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c6e4-23bf-43e1-81d9-d6f14807611c service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.866218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b17933-9028-4dcf-9649-0b518d46fd58 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.871921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.872739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.873023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.873296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.873399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.873937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.874204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.874312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.874462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.875116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.876213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.876851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.888991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 972/1966] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:55.893247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.904134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228399f0-1cc1-4a1b-b4f7-d25e57df4924 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.909932 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 995/1967] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:55.911190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:55.911763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97659c30-fb00-4d6f-a78b-3b420c67f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:55.912666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97659c30-fb00-4d6f-a78b-3b420c67f895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.912945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97659c30-fb00-4d6f-a78b-3b420c67f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.913216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97659c30-fb00-4d6f-a78b-3b420c67f895 admin admin] PATH_INFO: `/v3/projects/984935c4f13b4faa8cc18f56535e31a0/users/aff1d3657c8843649c6496622d215836/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.915129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97659c30-fb00-4d6f-a78b-3b420c67f895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984935c4f13b4faa8cc18f56535e31a0, user_id=aff1d3657c8843649c6496622d215836, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:55.917899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97659c30-fb00-4d6f-a78b-3b420c67f895 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:55.923029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bda273-e997-47ce-8235-f4b68a2520fc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:55.929597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 973/1968] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:55 2022] PUT /identity/v3/projects/984935c4f13b4faa8cc18f56535e31a0/users/aff1d3657c8843649c6496622d215836/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:55.947300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:55.948008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:55.948298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:55.983620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] MFA Rules not processed for user `aff1d3657c8843649c6496622d215836`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:55.985687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.006722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.007670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.008116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.008565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.009996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.013451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.026426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 996/1969] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:56.030588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.035375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:56.043533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce779b-9ce9-4fca-bbf9-12f96f4e1bea admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.050255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.050881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.051159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.051488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.052388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.054630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.070881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 997/1970] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:56.073727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 974/1971] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:56.074744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.085290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.085877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745265e9-e13f-492f-afea-cdda74ff8beb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.086817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.090763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.091442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.091722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.091995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] PATH_INFO: `/v3/projects/17055643a8d444b48a343ea95ee61bbd/users/8a7863895c8848258368f13ffef7812d/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.112606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=17055643a8d444b48a343ea95ee61bbd, user_id=8a7863895c8848258368f13ffef7812d, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.115297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.132187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 998/1972] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:56 2022] PUT /identity/v3/projects/17055643a8d444b48a343ea95ee61bbd/users/8a7863895c8848258368f13ffef7812d/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:56.133092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.134733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.160763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee179-efd1-439a-8b41-55d9b56dfbe7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.205135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:56.211106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b1eca-c224-4de6-9253-2082250ac2f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.211729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b1eca-c224-4de6-9253-2082250ac2f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.212007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b1eca-c224-4de6-9253-2082250ac2f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.212278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b1eca-c224-4de6-9253-2082250ac2f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.219416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff4520-cbf2-42de-906c-7722b2823abf admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.224082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.224751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.225065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.225338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] PATH_INFO: `/v3/projects/3119ffe3f67d48e9b994463590d95f6c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.229410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3119ffe3f67d48e9b994463590d95f6c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.232720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.282697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b1eca-c224-4de6-9253-2082250ac2f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.284480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b1eca-c224-4de6-9253-2082250ac2f5 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.286226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3119ffe3f67d48e9b994463590d95f6c', 'request_id': 'req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.294644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3119ffe3f67d48e9b994463590d95f6c', 'request_id': 'req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.296458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 975/1973] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:56.297825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.298116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.298387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.301152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3119ffe3f67d48e9b994463590d95f6c', 'request_id': 'req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.315384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] The token cache is being invalidate because project 3119ffe3f67d48e9b994463590d95f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:56.315759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.316551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.318511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 999/1974] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:56 2022] DELETE /identity/v3/projects/3119ffe3f67d48e9b994463590d95f6c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:56.331217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.332778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.339543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] MFA Rules not processed for user `8a7863895c8848258368f13ffef7812d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:56.341246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.382210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:56.423538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ec8ec-6c00-4e40-9de4-31008a26d0f3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.426890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 976/1975] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:56.427740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.429155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.431941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.432958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.434007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.434527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] PATH_INFO: `/v3/users/60c823809e1d466083f790b779143c06` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.435958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=60c823809e1d466083f790b779143c06)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.438394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.447430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd02268-84bc-433a-9607-bc69a9921d1c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.454182 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.455082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.455395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.456115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] PATH_INFO: `/v3/users/aee14af6ca12448e8ff7fd2491573911` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.457364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee14af6ca12448e8ff7fd2491573911)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.459705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.516644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60c823809e1d466083f790b779143c06'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.524009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee14af6ca12448e8ff7fd2491573911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.526297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60c823809e1d466083f790b779143c06'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.527763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60c823809e1d466083f790b779143c06'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.533716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee14af6ca12448e8ff7fd2491573911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.534607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee14af6ca12448e8ff7fd2491573911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.536732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60c823809e1d466083f790b779143c06'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.543258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee14af6ca12448e8ff7fd2491573911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.553391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60c823809e1d466083f790b779143c06'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.557807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dcdc95bf-5bae-43cd-adc6-3e282d1cbecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee14af6ca12448e8ff7fd2491573911'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:56.562316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1000/1976] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:56 2022] DELETE /identity/v3/users/60c823809e1d466083f790b779143c06 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:56.562815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.564195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.564338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 977/1977] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:56 2022] DELETE /identity/v3/users/aee14af6ca12448e8ff7fd2491573911 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:56.569091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.569382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.569659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.581901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:56.589715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:56.591136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.650807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d521126-1d9e-4bd5-9914-1ae3dce7d8dc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.676611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 978/1978] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:56.683197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.684676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.720334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5502f58-3346-4dec-b2cb-a0c8f16d2306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.721005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5502f58-3346-4dec-b2cb-a0c8f16d2306 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.721313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5502f58-3346-4dec-b2cb-a0c8f16d2306 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.721851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5502f58-3346-4dec-b2cb-a0c8f16d2306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.723688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c5502f58-3346-4dec-b2cb-a0c8f16d2306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.753456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:56.773225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19a2a0-3f58-4598-ae20-4d4a73a9a994 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.777514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.778143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.778806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.778806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.789077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.790921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.800999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 979/1979] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:56.803816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5502f58-3346-4dec-b2cb-a0c8f16d2306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.805316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5502f58-3346-4dec-b2cb-a0c8f16d2306 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:56.814776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1001/1980] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:02:56.934666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:56.936278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:56.950699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f39c70-40bb-4208-981a-c9d61b57f377 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:56.957311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:56.957921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:56.958116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:56.958423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] PATH_INFO: `/v3/projects/71d6a02b7bfa459e838d8d76c061ccc9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:56.959668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d6a02b7bfa459e838d8d76c061ccc9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:56.963680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.025455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d6a02b7bfa459e838d8d76c061ccc9', 'request_id': 'req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.037421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d6a02b7bfa459e838d8d76c061ccc9', 'request_id': 'req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.046010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d6a02b7bfa459e838d8d76c061ccc9', 'request_id': 'req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.058989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] The token cache is being invalidate because project 71d6a02b7bfa459e838d8d76c061ccc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:57.059301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.061190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be5f3091-8a79-4c98-b5b9-ef529a5b64ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.063283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 980/1981] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:56 2022] DELETE /identity/v3/projects/71d6a02b7bfa459e838d8d76c061ccc9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:57.068656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.069144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.069426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.092524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:57.093775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.173712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1002/1982] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:57.182431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc1879-08ba-4f8a-91b8-2df5889fc081 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.182673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc1879-08ba-4f8a-91b8-2df5889fc081 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.182976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc1879-08ba-4f8a-91b8-2df5889fc081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.208754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.209460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1ffc1879-08ba-4f8a-91b8-2df5889fc081 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:57.210212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.211107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1ffc1879-08ba-4f8a-91b8-2df5889fc081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.221809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 981/1983] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:57.226185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc7c4b-6d7d-45a6-b1c1-5bbae7c0b3e8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.226345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc1879-08ba-4f8a-91b8-2df5889fc081 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.232011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.232876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.233152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.233415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] PATH_INFO: `/v3/projects/0c1f844a3b0e4db0a20b30ab171cfdcd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.236765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc1879-08ba-4f8a-91b8-2df5889fc081 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.241811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.242087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c1f844a3b0e4db0a20b30ab171cfdcd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.242545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.242876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.243319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.244014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.244136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:57.244385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.246306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.252981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:57.260465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 982/1984] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:02:57.264211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.274167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169df923-0d93-4838-ab3a-6496f97d2b5d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.280292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.280965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.281239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.281504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.282164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.284044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.293894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c1f844a3b0e4db0a20b30ab171cfdcd', 'request_id': 'req-7c41a927-ec60-4399-9fdb-b393dbcf7132'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.303519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c1f844a3b0e4db0a20b30ab171cfdcd', 'request_id': 'req-7c41a927-ec60-4399-9fdb-b393dbcf7132'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.309786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c1f844a3b0e4db0a20b30ab171cfdcd', 'request_id': 'req-7c41a927-ec60-4399-9fdb-b393dbcf7132'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.315570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 983/1985] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:57 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:57.316562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.322426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] The token cache is being invalidate because project 0c1f844a3b0e4db0a20b30ab171cfdcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:57.322766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.323370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.325094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1003/1986] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:57 2022] DELETE /identity/v3/projects/0c1f844a3b0e4db0a20b30ab171cfdcd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:57.328779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.330726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.382617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac12d0-aabd-4598-9ff6-29e4b73e7b45 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.388336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.389383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.389383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.389633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.390462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.393297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.397962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41a927-ec60-4399-9fdb-b393dbcf7132 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.402904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.403636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.403930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.404203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] PATH_INFO: `/v3/users/aff1d3657c8843649c6496622d215836` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.405551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff1d3657c8843649c6496622d215836)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.407700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.416614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 984/1987] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:57 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:57.420706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.422301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.435014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21752a3-b1af-4bac-9c81-f3b350f77b28 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.440146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.440945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.441220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.441487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.442134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.444015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.454904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 985/1988] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:57.459037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.461777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff1d3657c8843649c6496622d215836'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.468856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0e84a-a933-4d46-a35a-4bcf0d2b2d3f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.470545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff1d3657c8843649c6496622d215836'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.471165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff1d3657c8843649c6496622d215836'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.474005 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.474581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.474855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.475184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.475822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.477611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.478574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff1d3657c8843649c6496622d215836'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.478934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:57.489837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 986/1989] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:57.493423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2ee3e8b0-12d4-4844-a2e8-aa37f22466de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff1d3657c8843649c6496622d215836'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:57.494609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.497458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.498279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1004/1990] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:02:57 2022] DELETE /identity/v3/users/aff1d3657c8843649c6496622d215836 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:57.513110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56648066-3a1c-4542-8d96-8250aa431b7d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.584497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.584950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.585230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.585505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] PATH_INFO: `/v3/projects/6498bbe6a7d342d48b0c4471d65ffe2f/users/913e95dee9d0463fa0b4bf8c54a63214/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.604558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6498bbe6a7d342d48b0c4471d65ffe2f, user_id=913e95dee9d0463fa0b4bf8c54a63214, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.607415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.622858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 987/1991] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:57 2022] PUT /identity/v3/projects/6498bbe6a7d342d48b0c4471d65ffe2f/users/913e95dee9d0463fa0b4bf8c54a63214/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:57.638070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.638396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.638668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.674082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] MFA Rules not processed for user `913e95dee9d0463fa0b4bf8c54a63214`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:57.675333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.680002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:57.763315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1005/1992] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:57.768459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.817333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.819291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:57.850201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accbd80-b18a-44ce-be29-c54865e1bd3b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.855570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.856466 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.856621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.856843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.857494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.859718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.896687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 988/1993] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:57 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:57.900799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:57.903326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b59ec-3f9b-44b1-a45b-6fb1c6c45265 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.967337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535b8de-c76d-4cd8-831c-a221b942f954 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:57.967909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.968540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.968831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.969115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] PATH_INFO: `/v3/projects/984935c4f13b4faa8cc18f56535e31a0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.970311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] RBAC: Authorizing `identity:delete_project(project_id=984935c4f13b4faa8cc18f56535e31a0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.972056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:57.972308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.972694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:57.972966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:57.973231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:57.974046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:57.976468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:57.987757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:58.005907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 989/1994] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:57 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:58.010518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.021270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ac51-204b-46d4-9364-b5766fbdab4c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.025215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984935c4f13b4faa8cc18f56535e31a0', 'request_id': 'req-530c2376-da41-4c6c-afdd-246932fb387c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:58.028941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:58.034097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984935c4f13b4faa8cc18f56535e31a0', 'request_id': 'req-530c2376-da41-4c6c-afdd-246932fb387c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:58.041652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984935c4f13b4faa8cc18f56535e31a0', 'request_id': 'req-530c2376-da41-4c6c-afdd-246932fb387c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:58.053157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] The token cache is being invalidate because project 984935c4f13b4faa8cc18f56535e31a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:02:58.053459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:58.054197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-530c2376-da41-4c6c-afdd-246932fb387c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:02:58.055965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1006/1995] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:02:57 2022] DELETE /identity/v3/projects/984935c4f13b4faa8cc18f56535e31a0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:58.060575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a27e-a373-4131-acf6-12a0d44ea91d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.060858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a27e-a373-4131-acf6-12a0d44ea91d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.061121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a27e-a373-4131-acf6-12a0d44ea91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.080174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-00f8a27e-a373-4131-acf6-12a0d44ea91d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:58.081067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.081419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-00f8a27e-a373-4131-acf6-12a0d44ea91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.081687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.081966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.082237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.082888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.084601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.089339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1007/1996] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:58.095074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 990/1997] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:58.097262 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dac5e4-111a-4ebd-ac82-42d8e37b62e4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.097544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dac5e4-111a-4ebd-ac82-42d8e37b62e4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.097847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dac5e4-111a-4ebd-ac82-42d8e37b62e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.099118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.100941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.113287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f4444-ffed-4733-a464-88b68aa7fa45 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.118230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.118669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-83dac5e4-111a-4ebd-ac82-42d8e37b62e4 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:58.118893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.119167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.119444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] PATH_INFO: `/v3/projects/0fe168f733904325a58649c12f5b35d4/users/74cc5ace47e94a129b90c29c69ea70ca/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.119846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-83dac5e4-111a-4ebd-ac82-42d8e37b62e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.128641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1008/1998] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:58.133065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dac5e4-111a-4ebd-ac82-42d8e37b62e4 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.135527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe168f733904325a58649c12f5b35d4, user_id=74cc5ace47e94a129b90c29c69ea70ca, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.138240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.142860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dac5e4-111a-4ebd-ac82-42d8e37b62e4 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.149636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.150260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.150537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.150808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.151460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:02:58.151757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.153151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 991/1999] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:58 2022] PUT /identity/v3/projects/0fe168f733904325a58649c12f5b35d4/users/74cc5ace47e94a129b90c29c69ea70ca/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:58.156540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.157489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.166384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1009/2000] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:02:58.170701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.181665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763489c-c57b-4672-92e3-f9cffdaef579 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.245285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750b490-d8a2-4efa-8844-5fd6911adb1b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.247575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.248200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.248531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.248832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.249471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.251306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.251438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.251922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.252195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.252969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.253626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.255233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.266885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 992/2001] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:58.271608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.282691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1010/2002] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:58.285238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0c2c0-d550-4c74-b1fe-b1e153ecaced admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.292911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.354716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c6519-8da5-4a7d-b34c-aa05af1e92c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.355554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c6519-8da5-4a7d-b34c-aa05af1e92c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.355818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c6519-8da5-4a7d-b34c-aa05af1e92c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.356096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c6519-8da5-4a7d-b34c-aa05af1e92c0 admin admin] PATH_INFO: `/v3/projects/0fe168f733904325a58649c12f5b35d4/users/74cc5ace47e94a129b90c29c69ea70ca/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.358166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c6519-8da5-4a7d-b34c-aa05af1e92c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe168f733904325a58649c12f5b35d4, user_id=74cc5ace47e94a129b90c29c69ea70ca, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.361304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c6519-8da5-4a7d-b34c-aa05af1e92c0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.364985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:58.380190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddb3c3-6d46-4cba-88b8-c449b69cc223 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.382372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 993/2003] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:58 2022] PUT /identity/v3/projects/0fe168f733904325a58649c12f5b35d4/users/74cc5ace47e94a129b90c29c69ea70ca/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:58.394867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.395511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.395809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.436558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:58.439209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] MFA Rules not processed for user `74cc5ace47e94a129b90c29c69ea70ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:58.440780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.452853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.453502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.453760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.454030 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.454671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.456679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.482889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1011/2004] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:58.486243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.502670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a2e617-7714-40fd-a6da-86ec16f853c2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.509254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.509955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.510232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.510506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.511165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.512837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.527104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1012/2005] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:58.527250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 994/2006] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:58.528194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.539380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.540076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c05585-bfe6-4662-8cbd-dc998b0ee7c8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.541109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.545602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.546396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.546666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.546934 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.548160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.549792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.562736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1013/2007] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:58.567201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.581268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990417d-e2fc-42e3-ac6e-36babdeb47bb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.586961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70e63b-731e-466c-b7c0-8890c25191aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.588078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70e63b-731e-466c-b7c0-8890c25191aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.588354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70e63b-731e-466c-b7c0-8890c25191aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.588648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70e63b-731e-466c-b7c0-8890c25191aa admin admin] PATH_INFO: `/v3/projects/463e6130d4cb463bbef9ebf8dcbe8877/users/c7705ff2b33048b9b57a6c056d6835d3/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.604551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70e63b-731e-466c-b7c0-8890c25191aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=463e6130d4cb463bbef9ebf8dcbe8877, user_id=c7705ff2b33048b9b57a6c056d6835d3, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.607484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8e1a3-e60d-41e5-a397-b180595c272d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.608521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70e63b-731e-466c-b7c0-8890c25191aa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.611793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.612260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.612577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.612984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.624034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.625373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1014/2008] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:58 2022] PUT /identity/v3/projects/463e6130d4cb463bbef9ebf8dcbe8877/users/c7705ff2b33048b9b57a6c056d6835d3/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:02:58.625649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.637993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7e61-1491-44ed-9663-e656b27bf8e8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.638210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7e61-1491-44ed-9663-e656b27bf8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.638431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7e61-1491-44ed-9663-e656b27bf8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.639569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 995/2009] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:58 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:58.669723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4a2a7e61-1491-44ed-9663-e656b27bf8e8 None None] MFA Rules not processed for user `c7705ff2b33048b9b57a6c056d6835d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:58.671760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4a2a7e61-1491-44ed-9663-e656b27bf8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.721250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.723170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.747527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dea2e-90a9-41a3-b6a8-45190227f5ba service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.789930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1015/2010] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:02:58.790928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a7e61-1491-44ed-9663-e656b27bf8e8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.818130 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.819211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.819492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.819765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.821823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:58.874221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a7e61-1491-44ed-9663-e656b27bf8e8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:58.883169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:58.884233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:58.884664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:58.884914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:58.885719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.887809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.904631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:58.937316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1016/2011] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:02:58.942377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:58.942528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:58.945190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:58.954719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:58.963405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 996/2012] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:59.035849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8713088-10cb-4a8c-9ae5-5eea51140b92 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:59.047685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:59.049138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.049138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.049138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.049762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:59.051749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:59.086123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1017/2013] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:02:58 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:02:59.089818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:59.101418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214d415-4add-4164-8bfe-7f61c0e8d104 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:59.106210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:59.106853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.107602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.107719 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.108403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:59.110269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:59.122597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 997/2014] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:02:59.126689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:59.139074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43c6e-1e59-49a1-b5f7-90157ec1e82a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:59.145100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:59.146410 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.146788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.147231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] PATH_INFO: `/v3/projects/f9abdf25ff864b10b4c97a195246252a/users/1484f0a2a371428a92feac792a6b8022/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.168754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9abdf25ff864b10b4c97a195246252a, user_id=1484f0a2a371428a92feac792a6b8022, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:59.171853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:59.192129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1018/2015] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:59 2022] PUT /identity/v3/projects/f9abdf25ff864b10b4c97a195246252a/users/1484f0a2a371428a92feac792a6b8022/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:59.192696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:59.203701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0593e-f994-40df-bef6-ce3fe2cbfafd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:59.287492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85c4be-b4d4-4fbc-9f7f-f1f3ae9742ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:59.289295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85c4be-b4d4-4fbc-9f7f-f1f3ae9742ce admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.289579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85c4be-b4d4-4fbc-9f7f-f1f3ae9742ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.289849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85c4be-b4d4-4fbc-9f7f-f1f3ae9742ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.290506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85c4be-b4d4-4fbc-9f7f-f1f3ae9742ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:59.292219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85c4be-b4d4-4fbc-9f7f-f1f3ae9742ce admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:59.309169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 998/2016] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:02:59.313711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:59.325910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414780e-7ea7-4467-a73b-1b2dd9097be7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:59.331411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:59.332777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.333060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.333350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] PATH_INFO: `/v3/projects/f9abdf25ff864b10b4c97a195246252a/users/1484f0a2a371428a92feac792a6b8022/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.335474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9abdf25ff864b10b4c97a195246252a, user_id=1484f0a2a371428a92feac792a6b8022, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:59.339818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:59.350903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1019/2017] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:02:59 2022] PUT /identity/v3/projects/f9abdf25ff864b10b4c97a195246252a/users/1484f0a2a371428a92feac792a6b8022/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:02:59.361818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5352-8c94-4107-b90f-bea433a8232f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.362116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5352-8c94-4107-b90f-bea433a8232f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.362390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5352-8c94-4107-b90f-bea433a8232f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.390878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1c0d5352-8c94-4107-b90f-bea433a8232f None None] MFA Rules not processed for user `1484f0a2a371428a92feac792a6b8022`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:02:59.392196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1c0d5352-8c94-4107-b90f-bea433a8232f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:02:59.464182 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 999/2018] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:02:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:02:59.476260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:59.533398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dffa8-281b-48b9-8cd1-9ccc7c3c46fd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:59.537541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:59.538185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.538463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.538732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.548361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:59.549843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:59.559613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1020/2019] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:02:59.658615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5352-8c94-4107-b90f-bea433a8232f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:02:59.671191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5352-8c94-4107-b90f-bea433a8232f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:02:59.676279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e879aa-c01c-455a-a38e-d54f1ed75e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:02:59.676920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e879aa-c01c-455a-a38e-d54f1ed75e23 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:02:59.677194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e879aa-c01c-455a-a38e-d54f1ed75e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:02:59.677434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e879aa-c01c-455a-a38e-d54f1ed75e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:02:59.726198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:02:59.742273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e879aa-c01c-455a-a38e-d54f1ed75e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:02:59.744684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e879aa-c01c-455a-a38e-d54f1ed75e23 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:02:59.756830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1000/2020] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:02:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:00.343080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:00.344787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:00.430844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b2a41-bb6c-4b7b-8bfc-2e48139b0901 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:00.436661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:00.437316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:00.437698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:00.438521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] PATH_INFO: `/v3/users/26c891bf39de4abfaa3a6e3aec5b90ea` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:00.439704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c891bf39de4abfaa3a6e3aec5b90ea)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:00.441728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:00.444534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:00.499728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c891bf39de4abfaa3a6e3aec5b90ea'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:00.508148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c891bf39de4abfaa3a6e3aec5b90ea'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:00.510858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c891bf39de4abfaa3a6e3aec5b90ea'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:00.522233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c891bf39de4abfaa3a6e3aec5b90ea'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:00.535186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c891bf39de4abfaa3a6e3aec5b90ea'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:00.541563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1021/2021] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:00 2022] DELETE /identity/v3/users/26c891bf39de4abfaa3a6e3aec5b90ea => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:00.546220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:00.546704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:00.546912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:00.566600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:00.567872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:00.632204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1001/2022] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:00.643998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:00.645623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:00.704562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee16ae-6052-4d26-9e1c-27627faf1a7e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:00.708757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005d314-42f0-42f4-8de3-cbb0ae590f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:00.709376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005d314-42f0-42f4-8de3-cbb0ae590f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:00.709720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005d314-42f0-42f4-8de3-cbb0ae590f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:00.709928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005d314-42f0-42f4-8de3-cbb0ae590f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:00.719669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005d314-42f0-42f4-8de3-cbb0ae590f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:00.721140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005d314-42f0-42f4-8de3-cbb0ae590f6d service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:00.730731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1022/2023] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:01.055534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.057571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:01.081506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b83c1-a18e-4f4e-afdd-c766ee6fe695 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:01.104596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:01.105399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.105545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.105825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] PATH_INFO: `/v3/projects/f1aa65f791484f6cb0b29da39d0e73f4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.107081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1aa65f791484f6cb0b29da39d0e73f4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:01.109218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:01.167787 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1aa65f791484f6cb0b29da39d0e73f4', 'request_id': 'req-ab630fcb-4e78-445c-bcf2-360fe58511e2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:01.178718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1aa65f791484f6cb0b29da39d0e73f4', 'request_id': 'req-ab630fcb-4e78-445c-bcf2-360fe58511e2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:01.188132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1aa65f791484f6cb0b29da39d0e73f4', 'request_id': 'req-ab630fcb-4e78-445c-bcf2-360fe58511e2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:01.212916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] The token cache is being invalidate because project f1aa65f791484f6cb0b29da39d0e73f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:01.213269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:01.214213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ab630fcb-4e78-445c-bcf2-360fe58511e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:01.215832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1002/2024] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:01 2022] DELETE /identity/v3/projects/f1aa65f791484f6cb0b29da39d0e73f4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:01.222778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd38be-c83c-46fc-8a5c-59e088f2cd9c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.222778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd38be-c83c-46fc-8a5c-59e088f2cd9c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.222778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd38be-c83c-46fc-8a5c-59e088f2cd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.247512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-edbd38be-c83c-46fc-8a5c-59e088f2cd9c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:01.248927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-edbd38be-c83c-46fc-8a5c-59e088f2cd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:01.352727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1023/2025] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:01.365661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66576b5-3ed6-4249-b916-1f3cc82019ac None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.366185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66576b5-3ed6-4249-b916-1f3cc82019ac None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.366859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66576b5-3ed6-4249-b916-1f3cc82019ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.402191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b66576b5-3ed6-4249-b916-1f3cc82019ac None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:01.404106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b66576b5-3ed6-4249-b916-1f3cc82019ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:01.421525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1003/2026] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:01.426145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd38be-c83c-46fc-8a5c-59e088f2cd9c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.440909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd38be-c83c-46fc-8a5c-59e088f2cd9c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:01.446516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:01.447153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.447436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.447710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.448389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:01.448733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:01.450931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:01.465860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1024/2027] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:01.471053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66576b5-3ed6-4249-b916-1f3cc82019ac None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.486773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66576b5-3ed6-4249-b916-1f3cc82019ac None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:01.494673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:01.495325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.495945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.496216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.496896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:01.499469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:01.541376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1004/2028] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:01 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:01.546109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.556829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba772ee-59cf-425c-a15a-fca8b754e79c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:01.566397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:01.631325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:01.632078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.632493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.632654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.638664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:01.638664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:01.668739 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1025/2029] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:01 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:01.674530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.692804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30095f59-bd28-459f-8033-157db77aeb85 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:01.698570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:01.699514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.699796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.700069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.700799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:01.702439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:01.713348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1005/2030] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:01.717713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.728759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22ed25-1c43-4d8e-9d37-892ef298f33c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:01.734525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc8938-963b-48d1-99ab-dc7d34ab394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:01.735365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8938-963b-48d1-99ab-dc7d34ab394b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.735650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8938-963b-48d1-99ab-dc7d34ab394b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.735900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8938-963b-48d1-99ab-dc7d34ab394b admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.736740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc8938-963b-48d1-99ab-dc7d34ab394b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:01.738498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc8938-963b-48d1-99ab-dc7d34ab394b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:01.750647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1026/2031] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:01.755922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.767649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da397043-8ab9-488e-b199-16e54a5cecd5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:01.772709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:01.773388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.773663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.773938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] PATH_INFO: `/v3/projects/aaa6db43343243a09e8ddc37f8f6c3e7/users/4b0a532615154eabb8d9ad74838b75dc/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.790495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa6db43343243a09e8ddc37f8f6c3e7, user_id=4b0a532615154eabb8d9ad74838b75dc, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:01.793302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:01.812994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1006/2032] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:01 2022] PUT /identity/v3/projects/aaa6db43343243a09e8ddc37f8f6c3e7/users/4b0a532615154eabb8d9ad74838b75dc/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:01.824403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:01.824728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:01.825004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:01.855485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] MFA Rules not processed for user `4b0a532615154eabb8d9ad74838b75dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:01.856748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:01.945902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1027/2033] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:01.957404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:01.959314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:01.970257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:02.021607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b9f0-8b37-44ad-97e1-d8a961c1d573 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:02.025461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf9649-7398-4328-9d9b-04486137856d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:02.026105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf9649-7398-4328-9d9b-04486137856d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:02.026377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf9649-7398-4328-9d9b-04486137856d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:02.026645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf9649-7398-4328-9d9b-04486137856d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:02.035779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf9649-7398-4328-9d9b-04486137856d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:02.037255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf9649-7398-4328-9d9b-04486137856d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:02.047855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1007/2034] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:02.631724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:02.633213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:02.711400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f9c4f-09cf-492e-ad69-85f4fffb5a9a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:02.716768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:02.717417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:02.717965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:02.718280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] PATH_INFO: `/v3/users/913e95dee9d0463fa0b4bf8c54a63214` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:02.719492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=913e95dee9d0463fa0b4bf8c54a63214)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:02.721512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:02.755229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:02.795854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913e95dee9d0463fa0b4bf8c54a63214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:02.804958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913e95dee9d0463fa0b4bf8c54a63214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:02.805761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913e95dee9d0463fa0b4bf8c54a63214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:02.813120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913e95dee9d0463fa0b4bf8c54a63214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:02.828552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913e95dee9d0463fa0b4bf8c54a63214'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:02.837668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1028/2035] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:02 2022] DELETE /identity/v3/users/913e95dee9d0463fa0b4bf8c54a63214 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:02.843674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:02.844156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:02.844545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:02.866662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:02.868125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:02.935110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1008/2036] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:02.945486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:02.947163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:03.013734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec495-153c-4386-b28d-248da69300b2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:03.018202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:03.018873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:03.019226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:03.019499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:03.030909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:03.032411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:03.042661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1029/2037] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:03.361914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:03.363934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:03.376234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4663eec-8bd1-47dd-b8b7-7356c6116a09 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:03.381788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:03.382572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:03.382949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:03.384287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] PATH_INFO: `/v3/projects/6498bbe6a7d342d48b0c4471d65ffe2f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:03.385276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6498bbe6a7d342d48b0c4471d65ffe2f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:03.387804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:03.476453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6498bbe6a7d342d48b0c4471d65ffe2f', 'request_id': 'req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.489269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6498bbe6a7d342d48b0c4471d65ffe2f', 'request_id': 'req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.498072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6498bbe6a7d342d48b0c4471d65ffe2f', 'request_id': 'req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.514350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] The token cache is being invalidate because project 6498bbe6a7d342d48b0c4471d65ffe2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:03.514752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.515468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.520222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1009/2038] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:03 2022] DELETE /identity/v3/projects/6498bbe6a7d342d48b0c4471d65ffe2f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:03.521599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:03.533600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bd5ca-9a72-4f84-bd14-18c69478d239 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:03.633858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:03.634623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:03.634978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:03.635438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] PATH_INFO: `/v3/users/74cc5ace47e94a129b90c29c69ea70ca` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:03.637269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74cc5ace47e94a129b90c29c69ea70ca)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:03.639679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:03.721459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:03.725901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74cc5ace47e94a129b90c29c69ea70ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.738985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74cc5ace47e94a129b90c29c69ea70ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.740276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74cc5ace47e94a129b90c29c69ea70ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.750547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74cc5ace47e94a129b90c29c69ea70ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.775558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf289b1b-99fc-40d3-9169-d8cd111f8696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74cc5ace47e94a129b90c29c69ea70ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:03.786172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1030/2039] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:03 2022] DELETE /identity/v3/users/74cc5ace47e94a129b90c29c69ea70ca => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:04.254340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:04.257227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:04.279472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf187bb-b31a-4dd0-a12d-dab75d4e7d78 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:04.311538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:04.341489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:04.342129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.342406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.342694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] PATH_INFO: `/v3/projects/0fe168f733904325a58649c12f5b35d4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.343840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe168f733904325a58649c12f5b35d4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:04.345803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:04.394067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe168f733904325a58649c12f5b35d4', 'request_id': 'req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:04.403373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe168f733904325a58649c12f5b35d4', 'request_id': 'req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:04.409903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe168f733904325a58649c12f5b35d4', 'request_id': 'req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:04.420455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] The token cache is being invalidate because project 0fe168f733904325a58649c12f5b35d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:04.420762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:04.421397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1f5bd6a7-ed55-4e1b-86e3-0e4bb5e83c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:04.422896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1010/2040] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:04 2022] DELETE /identity/v3/projects/0fe168f733904325a58649c12f5b35d4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:04.428001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f950f4-9beb-4a55-bde6-c270eff10bed None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.428333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f950f4-9beb-4a55-bde6-c270eff10bed None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.428635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f950f4-9beb-4a55-bde6-c270eff10bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.448796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-76f950f4-9beb-4a55-bde6-c270eff10bed None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:04.450144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-76f950f4-9beb-4a55-bde6-c270eff10bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:04.517270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1031/2041] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:04.528327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c20dbc-9edf-4afc-b205-f9342be5adef None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.528700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c20dbc-9edf-4afc-b205-f9342be5adef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.529054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c20dbc-9edf-4afc-b205-f9342be5adef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.559354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f9c20dbc-9edf-4afc-b205-f9342be5adef None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:04.560609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f9c20dbc-9edf-4afc-b205-f9342be5adef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:04.569836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1011/2042] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:04.574560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f950f4-9beb-4a55-bde6-c270eff10bed None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:04.585027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f950f4-9beb-4a55-bde6-c270eff10bed None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:04.590515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:04.591105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.591389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.591652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.592300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:04.592634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:04.594231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:04.606022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1032/2043] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:04.610536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c20dbc-9edf-4afc-b205-f9342be5adef None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:04.622631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c20dbc-9edf-4afc-b205-f9342be5adef None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:04.629607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:04.630225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.630498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.630764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.631395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:04.633341 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:04.667050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1012/2044] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:04 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:04.668361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:04.683544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec97025-de48-4f1d-8231-9828721b84c9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:04.740902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:04.741529 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.742277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.742277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.742721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:04.744767 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:04.767760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1033/2045] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:04 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:04.771892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:04.786982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ad90-d987-456f-bfea-a897e40d0266 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:04.792809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:04.793437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.793716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.793987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.794678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:04.796355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:04.807309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1013/2046] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:04.811347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:04.821709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94affad1-7482-4aaf-b789-b3208884266f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:04.826813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef65f08-1934-414b-a874-ce712b252f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:04.827426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65f08-1934-414b-a874-ce712b252f64 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.827812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65f08-1934-414b-a874-ce712b252f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.828090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65f08-1934-414b-a874-ce712b252f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.828953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef65f08-1934-414b-a874-ce712b252f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:04.830539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef65f08-1934-414b-a874-ce712b252f64 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:04.842660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1034/2047] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:04.847369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:04.857905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94539-94dc-41fc-a692-9ea612c51c85 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:04.863066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:04.863770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.864052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.864323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] PATH_INFO: `/v3/projects/6a9ca31edd5b47ec889d18e6d412ad59/users/605d4edcffb64b72b4551cb45eb0720d/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.879748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9ca31edd5b47ec889d18e6d412ad59, user_id=605d4edcffb64b72b4551cb45eb0720d, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:04.882587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:04.896770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1014/2048] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:04 2022] PUT /identity/v3/projects/6a9ca31edd5b47ec889d18e6d412ad59/users/605d4edcffb64b72b4551cb45eb0720d/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:04.908103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925d36a-8d4a-424f-9172-bc6915e46023 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:04.908417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925d36a-8d4a-424f-9172-bc6915e46023 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:04.908700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925d36a-8d4a-424f-9172-bc6915e46023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:04.936960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c925d36a-8d4a-424f-9172-bc6915e46023 None None] MFA Rules not processed for user `605d4edcffb64b72b4551cb45eb0720d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:04.938156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c925d36a-8d4a-424f-9172-bc6915e46023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:04.951962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:05.018612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1035/2049] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:05.023246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.093967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5468c3-2119-42dc-9163-33a80901ce43 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.098949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.099626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.099899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.100298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.100979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.102918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.108475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:05.142458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1015/2050] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:05 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:05.146586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925d36a-8d4a-424f-9172-bc6915e46023 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.159896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925d36a-8d4a-424f-9172-bc6915e46023 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.228240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.228889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.229167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.229436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.230067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.232017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.259181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1036/2051] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:05 2022] POST /identity/v3/users => generated 301 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:05.263378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.274186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89642b5e-4680-4ef2-84b4-ed6401205786 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.279335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.279951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.280222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.280596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.281256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.282873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.293994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1016/2052] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:05.298297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.309199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed70bef-2ad5-433d-b8a7-d3f88ded4906 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.314722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.315401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.315678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.315945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] PATH_INFO: `/v3/projects/bb3b54485df049049b1e0cc4b19d3d4c/users/9314f2068e6942028650f4df60fef802/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.331565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3b54485df049049b1e0cc4b19d3d4c, user_id=9314f2068e6942028650f4df60fef802, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.334249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.353561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1037/2053] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:05 2022] PUT /identity/v3/projects/bb3b54485df049049b1e0cc4b19d3d4c/users/9314f2068e6942028650f4df60fef802/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:05.357856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.368619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1746-ead2-4867-9e31-fef1a6033263 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.427391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15d139-f1aa-4470-9c3c-275b07a876b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.428023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15d139-f1aa-4470-9c3c-275b07a876b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.428296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15d139-f1aa-4470-9c3c-275b07a876b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.428583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15d139-f1aa-4470-9c3c-275b07a876b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.429206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15d139-f1aa-4470-9c3c-275b07a876b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.430863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15d139-f1aa-4470-9c3c-275b07a876b2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.441277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1017/2054] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:05.445546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.456850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a53e31-a7e6-41f5-a0e1-f5de83bfd7ac admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.462230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.462886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.463167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.463445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] PATH_INFO: `/v3/projects/bb3b54485df049049b1e0cc4b19d3d4c/users/9314f2068e6942028650f4df60fef802/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.465459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3b54485df049049b1e0cc4b19d3d4c, user_id=9314f2068e6942028650f4df60fef802, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.468108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.479187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1038/2055] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:05 2022] PUT /identity/v3/projects/bb3b54485df049049b1e0cc4b19d3d4c/users/9314f2068e6942028650f4df60fef802/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:05.489730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33692ce1-54b4-4290-97ad-6b497f5b7b1c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.490020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33692ce1-54b4-4290-97ad-6b497f5b7b1c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.490313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33692ce1-54b4-4290-97ad-6b497f5b7b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.519998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-33692ce1-54b4-4290-97ad-6b497f5b7b1c None None] MFA Rules not processed for user `9314f2068e6942028650f4df60fef802`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:05.521214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-33692ce1-54b4-4290-97ad-6b497f5b7b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:05.591209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1018/2056] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:05.602220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.604148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:05.669749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c02de-4660-4c12-aa50-cf2dcb1616b2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.674910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b974511-b538-454d-a2b5-fdbcfe4bd1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.675554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974511-b538-454d-a2b5-fdbcfe4bd1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.675825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974511-b538-454d-a2b5-fdbcfe4bd1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.676091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b974511-b538-454d-a2b5-fdbcfe4bd1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.686699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b974511-b538-454d-a2b5-fdbcfe4bd1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.688156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b974511-b538-454d-a2b5-fdbcfe4bd1e7 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.698171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1039/2057] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:05.851503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33692ce1-54b4-4290-97ad-6b497f5b7b1c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:05.915250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:05.921801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33692ce1-54b4-4290-97ad-6b497f5b7b1c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:05.927379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:05.928028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:05.928304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:05.928607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] PATH_INFO: `/v3/users/605d4edcffb64b72b4551cb45eb0720d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:05.929778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=605d4edcffb64b72b4551cb45eb0720d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:05.931826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:05.986030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605d4edcffb64b72b4551cb45eb0720d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:05.994588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605d4edcffb64b72b4551cb45eb0720d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:05.995194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605d4edcffb64b72b4551cb45eb0720d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.001933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605d4edcffb64b72b4551cb45eb0720d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.013762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605d4edcffb64b72b4551cb45eb0720d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.019556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1019/2058] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:05 2022] DELETE /identity/v3/users/605d4edcffb64b72b4551cb45eb0720d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:06.024316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:06.024668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:06.024940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:06.043861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:06.045090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:06.053291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:06.117733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1040/2059] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:06.129811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:06.131446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:06.201091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40844-1457-42ed-8ed8-7884f59bbc93 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:06.204883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:06.205505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:06.205783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:06.206055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:06.216330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:06.218075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:06.228173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1020/2060] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:06.518657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:06.520252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:06.533831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403b39f-cc5b-45e3-b049-3ce9c00b05ac None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:06.538649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:06.539315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:06.539594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:06.539914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] PATH_INFO: `/v3/projects/6a9ca31edd5b47ec889d18e6d412ad59` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:06.541211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a9ca31edd5b47ec889d18e6d412ad59)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:06.543059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:06.592804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a9ca31edd5b47ec889d18e6d412ad59', 'request_id': 'req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.600349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a9ca31edd5b47ec889d18e6d412ad59', 'request_id': 'req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.606208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a9ca31edd5b47ec889d18e6d412ad59', 'request_id': 'req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.617557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] The token cache is being invalidate because project 6a9ca31edd5b47ec889d18e6d412ad59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:06.617782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.618603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.620160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1041/2061] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:06 2022] DELETE /identity/v3/projects/6a9ca31edd5b47ec889d18e6d412ad59 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:06.624438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:06.634697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c1780-89b5-4cea-8f6a-e7f1e9ab685a service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:06.694558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:06.695214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:06.695484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:06.695761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] PATH_INFO: `/v3/users/9314f2068e6942028650f4df60fef802` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:06.697097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9314f2068e6942028650f4df60fef802)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:06.699045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:06.720038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:06.748696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9314f2068e6942028650f4df60fef802'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.756898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9314f2068e6942028650f4df60fef802'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.757633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9314f2068e6942028650f4df60fef802'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.764872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9314f2068e6942028650f4df60fef802'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.776834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a5b77933-5000-4c9e-97bc-9aed8004951d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9314f2068e6942028650f4df60fef802'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:06.782712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1021/2062] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:06 2022] DELETE /identity/v3/users/9314f2068e6942028650f4df60fef802 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:07.115300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.116241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:07.130890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e00330-d3fd-4d9c-8d74-01c8e47593ba admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.232408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.233101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.233379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.233649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] PATH_INFO: `/v3/projects/bb3b54485df049049b1e0cc4b19d3d4c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.234828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3b54485df049049b1e0cc4b19d3d4c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.236824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.278278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:07.288908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3b54485df049049b1e0cc4b19d3d4c', 'request_id': 'req-72d08fef-c9af-4a8a-9409-82df89d8a00e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:07.296886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3b54485df049049b1e0cc4b19d3d4c', 'request_id': 'req-72d08fef-c9af-4a8a-9409-82df89d8a00e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:07.302705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3b54485df049049b1e0cc4b19d3d4c', 'request_id': 'req-72d08fef-c9af-4a8a-9409-82df89d8a00e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:07.313781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] The token cache is being invalidate because project bb3b54485df049049b1e0cc4b19d3d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:07.314072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:07.314719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-72d08fef-c9af-4a8a-9409-82df89d8a00e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:07.316243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1042/2063] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:07 2022] DELETE /identity/v3/projects/bb3b54485df049049b1e0cc4b19d3d4c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:07.321480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc559329-9fa3-451c-bfc9-b471643d0695 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.321821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc559329-9fa3-451c-bfc9-b471643d0695 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.322083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc559329-9fa3-451c-bfc9-b471643d0695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.342926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cc559329-9fa3-451c-bfc9-b471643d0695 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:07.344150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cc559329-9fa3-451c-bfc9-b471643d0695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:07.408056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1022/2064] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:07.417220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c47f7-6587-4c5c-8b52-0ef03ba0ecf2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.417480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c47f7-6587-4c5c-8b52-0ef03ba0ecf2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.417735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c47f7-6587-4c5c-8b52-0ef03ba0ecf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.438079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ca4c47f7-6587-4c5c-8b52-0ef03ba0ecf2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:07.439298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ca4c47f7-6587-4c5c-8b52-0ef03ba0ecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:07.447404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1043/2065] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:07.451765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc559329-9fa3-451c-bfc9-b471643d0695 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.461811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc559329-9fa3-451c-bfc9-b471643d0695 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.466610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.467216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.467485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.467750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.468406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:07.468729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.470336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.489003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1023/2066] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:07.493705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c47f7-6587-4c5c-8b52-0ef03ba0ecf2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.504140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c47f7-6587-4c5c-8b52-0ef03ba0ecf2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.509086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.509703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.510005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.510252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.510892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.512860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.541989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1044/2067] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:07 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:07.546223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.556724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7c095-0f2e-4103-8761-7f184749271e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.614983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.615601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.615873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.616135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.616825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.618873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.641796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1024/2068] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:07 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:07.645978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.657297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822fb07-9ecb-4f5e-9fad-f72d2a211d18 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.662122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.662727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.663001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.663268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.663910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.665656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.675745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1045/2069] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:07.680000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.690823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced82deb-a555-4a3e-b4b5-d6cff10fc8d2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.696193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c565e58-4d5f-455e-98bd-37d53ed42b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.696837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c565e58-4d5f-455e-98bd-37d53ed42b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.697122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c565e58-4d5f-455e-98bd-37d53ed42b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.697385 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c565e58-4d5f-455e-98bd-37d53ed42b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.698044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c565e58-4d5f-455e-98bd-37d53ed42b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.699669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c565e58-4d5f-455e-98bd-37d53ed42b0f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.710881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1025/2070] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:07.715184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.725492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a44639-81d5-47d7-9324-f32c50082411 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.730353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.731024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.731298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.731568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] PATH_INFO: `/v3/projects/a607d59e77604a02bb2058c1f7af225d/users/022954913ef04c51b2a634905b9883d1/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.747546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a607d59e77604a02bb2058c1f7af225d, user_id=022954913ef04c51b2a634905b9883d1, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.750248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.764606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1046/2071] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:07 2022] PUT /identity/v3/projects/a607d59e77604a02bb2058c1f7af225d/users/022954913ef04c51b2a634905b9883d1/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:07.775043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09a9ac-6628-43ae-95fe-e292368f1de0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.775328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09a9ac-6628-43ae-95fe-e292368f1de0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.775588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09a9ac-6628-43ae-95fe-e292368f1de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.782909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:07.805865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bb09a9ac-6628-43ae-95fe-e292368f1de0 None None] MFA Rules not processed for user `022954913ef04c51b2a634905b9883d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:07.807089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb09a9ac-6628-43ae-95fe-e292368f1de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:07.872399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1026/2072] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:07.878335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.942607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb015c-7cfe-40b1-9f83-fbf6401790f3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:07.947863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:07.948541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:07.948949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:07.949068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:07.949728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:07.951668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:07.982403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1047/2073] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:07 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:07.987118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09a9ac-6628-43ae-95fe-e292368f1de0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:07.999384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09a9ac-6628-43ae-95fe-e292368f1de0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:08.067547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:08.068202 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:08.068504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:08.068776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:08.069423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:08.071375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:08.095837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1027/2074] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:07 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:08.100048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:08.121061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b7466-1265-415c-87cc-5fc1c4c9ef76 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:08.127600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:08.128238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:08.128666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:08.128823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:08.129451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:08.131095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:08.142702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1048/2075] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:08.147321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:08.158159 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284cfe7c-69bd-4c63-a228-606698098de9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:08.162973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:08.163644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:08.163920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:08.164215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] PATH_INFO: `/v3/projects/a7514eb7951f42b5b266fbdb8a29e7ea/users/c13d42ca38554a2aa219bf9f76b787f6/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:08.182278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7514eb7951f42b5b266fbdb8a29e7ea, user_id=c13d42ca38554a2aa219bf9f76b787f6, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:08.186654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:08.209553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1028/2076] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:08 2022] PUT /identity/v3/projects/a7514eb7951f42b5b266fbdb8a29e7ea/users/c13d42ca38554a2aa219bf9f76b787f6/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:08.214019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:08.239064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e6868-4358-4520-ac62-40cae8ef63fa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:08.275503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:08.314354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac53a6-24cf-4ab6-8d5b-c4277d2699b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:08.315794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac53a6-24cf-4ab6-8d5b-c4277d2699b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:08.316063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac53a6-24cf-4ab6-8d5b-c4277d2699b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:08.316338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac53a6-24cf-4ab6-8d5b-c4277d2699b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:08.317194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac53a6-24cf-4ab6-8d5b-c4277d2699b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:08.319269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac53a6-24cf-4ab6-8d5b-c4277d2699b4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:08.338395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1049/2077] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:08.340809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:08.354608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59d71-cce7-410b-9c66-1ae8b09ac6c7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:08.367316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:08.368716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:08.369063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:08.369345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] PATH_INFO: `/v3/projects/a7514eb7951f42b5b266fbdb8a29e7ea/users/c13d42ca38554a2aa219bf9f76b787f6/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:08.374562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7514eb7951f42b5b266fbdb8a29e7ea, user_id=c13d42ca38554a2aa219bf9f76b787f6, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:08.378025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:08.396937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1029/2078] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:08 2022] PUT /identity/v3/projects/a7514eb7951f42b5b266fbdb8a29e7ea/users/c13d42ca38554a2aa219bf9f76b787f6/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:08.407428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2a6f8-1d4f-4257-bb30-168e729f0238 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:08.407721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2a6f8-1d4f-4257-bb30-168e729f0238 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:08.408001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2a6f8-1d4f-4257-bb30-168e729f0238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:08.437708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-33e2a6f8-1d4f-4257-bb30-168e729f0238 None None] MFA Rules not processed for user `c13d42ca38554a2aa219bf9f76b787f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:08.438775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-33e2a6f8-1d4f-4257-bb30-168e729f0238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:08.542415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1050/2079] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:08.553900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:08.556311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:08.634740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:08.649210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87207c-c58c-4194-a613-442905ce6333 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:08.658703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b01db-f866-439e-9831-256cee15483e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:08.659867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b01db-f866-439e-9831-256cee15483e service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:08.660346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b01db-f866-439e-9831-256cee15483e service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:08.660843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b01db-f866-439e-9831-256cee15483e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:08.688490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b01db-f866-439e-9831-256cee15483e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:08.689269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b01db-f866-439e-9831-256cee15483e service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:08.700948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1030/2080] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:09.199233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2a6f8-1d4f-4257-bb30-168e729f0238 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:09.274688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2a6f8-1d4f-4257-bb30-168e729f0238 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:09.279401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:09.280079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:09.280361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:09.280747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] PATH_INFO: `/v3/users/022954913ef04c51b2a634905b9883d1` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:09.281903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=022954913ef04c51b2a634905b9883d1)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:09.283817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:09.334488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '022954913ef04c51b2a634905b9883d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.344273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '022954913ef04c51b2a634905b9883d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.345395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '022954913ef04c51b2a634905b9883d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.353653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:09.357181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '022954913ef04c51b2a634905b9883d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.372635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '022954913ef04c51b2a634905b9883d1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.379955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1051/2081] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:09 2022] DELETE /identity/v3/users/022954913ef04c51b2a634905b9883d1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:09.384895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957089-1074-45e1-924a-b20748368f0c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:09.385148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957089-1074-45e1-924a-b20748368f0c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:09.385516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957089-1074-45e1-924a-b20748368f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:09.405459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-72957089-1074-45e1-924a-b20748368f0c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:09.406716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72957089-1074-45e1-924a-b20748368f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:09.469206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1031/2082] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:09.479620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:09.481214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:09.545391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7465b-69f6-4efa-8d52-0f40c5baa436 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:09.549175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:09.549779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:09.550056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:09.550324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:09.560415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:09.561889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:09.571589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1052/2083] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:09.863876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957089-1074-45e1-924a-b20748368f0c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:09.866322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72957089-1074-45e1-924a-b20748368f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:09.885122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957089-1074-45e1-924a-b20748368f0c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:09.890690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:09.891347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:09.891620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:09.891933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] PATH_INFO: `/v3/projects/a607d59e77604a02bb2058c1f7af225d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:09.893665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a607d59e77604a02bb2058c1f7af225d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:09.895834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:09.946403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a607d59e77604a02bb2058c1f7af225d', 'request_id': 'req-a6155540-0ee9-4de9-9dfe-3a021f540a1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.956013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a607d59e77604a02bb2058c1f7af225d', 'request_id': 'req-a6155540-0ee9-4de9-9dfe-3a021f540a1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.962726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a607d59e77604a02bb2058c1f7af225d', 'request_id': 'req-a6155540-0ee9-4de9-9dfe-3a021f540a1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.973657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] The token cache is being invalidate because project a607d59e77604a02bb2058c1f7af225d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:09.973950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.974675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:09.976490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1032/2084] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:09 2022] DELETE /identity/v3/projects/a607d59e77604a02bb2058c1f7af225d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:09.980440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:09.993924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe044b3a-13ab-4b01-8fd0-446b688dbbab service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:10.052072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:10.052748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.053027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.053297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] PATH_INFO: `/v3/users/c13d42ca38554a2aa219bf9f76b787f6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.054563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13d42ca38554a2aa219bf9f76b787f6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:10.056602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:10.104537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13d42ca38554a2aa219bf9f76b787f6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.113203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13d42ca38554a2aa219bf9f76b787f6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.113860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13d42ca38554a2aa219bf9f76b787f6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.121485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13d42ca38554a2aa219bf9f76b787f6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.136699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-50e9670e-1e9d-4920-b69e-cf002eb48954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13d42ca38554a2aa219bf9f76b787f6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.138384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:10.144752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1053/2085] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:09 2022] DELETE /identity/v3/users/c13d42ca38554a2aa219bf9f76b787f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:10.437685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:10.439256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:10.464820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6155540-0ee9-4de9-9dfe-3a021f540a1d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:10.525300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:10.525940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.526214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.526485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] PATH_INFO: `/v3/projects/a7514eb7951f42b5b266fbdb8a29e7ea` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.527597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7514eb7951f42b5b266fbdb8a29e7ea)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:10.529610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:10.541525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:10.579082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7514eb7951f42b5b266fbdb8a29e7ea', 'request_id': 'req-ff5717f8-15ca-47e8-ac8d-d15006271151'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.587588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7514eb7951f42b5b266fbdb8a29e7ea', 'request_id': 'req-ff5717f8-15ca-47e8-ac8d-d15006271151'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.598948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7514eb7951f42b5b266fbdb8a29e7ea', 'request_id': 'req-ff5717f8-15ca-47e8-ac8d-d15006271151'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.612497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] The token cache is being invalidate because project a7514eb7951f42b5b266fbdb8a29e7ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:10.612791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.613470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ff5717f8-15ca-47e8-ac8d-d15006271151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:10.615035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1033/2086] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:10 2022] DELETE /identity/v3/projects/a7514eb7951f42b5b266fbdb8a29e7ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:10.620746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01b481-1385-452d-8a9b-42448d224d58 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.621238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01b481-1385-452d-8a9b-42448d224d58 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.621435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01b481-1385-452d-8a9b-42448d224d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.654762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-be01b481-1385-452d-8a9b-42448d224d58 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:10.656035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-be01b481-1385-452d-8a9b-42448d224d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:10.722477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1054/2087] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:10.732113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913bc3d-a0c1-41b1-9699-6e109ef1daec None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.732458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913bc3d-a0c1-41b1-9699-6e109ef1daec None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.732729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913bc3d-a0c1-41b1-9699-6e109ef1daec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.755098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0913bc3d-a0c1-41b1-9699-6e109ef1daec None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:10.756309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0913bc3d-a0c1-41b1-9699-6e109ef1daec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:10.764857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1034/2088] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:10.769185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01b481-1385-452d-8a9b-42448d224d58 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:10.779505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01b481-1385-452d-8a9b-42448d224d58 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:10.784853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:10.785468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.785747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.786016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.786677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:10.786982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:10.788651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:10.801068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1055/2089] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:10.807643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913bc3d-a0c1-41b1-9699-6e109ef1daec None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:10.821737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913bc3d-a0c1-41b1-9699-6e109ef1daec None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:10.827368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:10.827968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.828243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.828553 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.829204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:10.831132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:10.862855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1035/2090] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:10 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:10.866924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:10.878652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166925a-8a12-4cf8-890c-02c2a1de1237 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:10.947795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:10.948616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.948953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.949301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.950057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:10.952231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:10.974845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1056/2091] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:10 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:10.979101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:10.990089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d3eba-08d7-45dd-963d-4ee3d83c8e76 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:10.994784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:10.995384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:10.995658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:10.995947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:10.996625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:10.998236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.012139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1036/2092] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:11.013706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.025793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262e864-7f6b-48a0-a1c5-be8fa69b1a0f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:11.033263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450dd00-1795-4589-86bb-7f95594bb36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:11.034206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450dd00-1795-4589-86bb-7f95594bb36e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.034542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450dd00-1795-4589-86bb-7f95594bb36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.035016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450dd00-1795-4589-86bb-7f95594bb36e admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.035613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450dd00-1795-4589-86bb-7f95594bb36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:11.037565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450dd00-1795-4589-86bb-7f95594bb36e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.050979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1057/2093] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:11.059540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.069910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b3d6d-65fe-4182-b83a-b8b224e90903 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:11.074898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:11.075862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.076310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.076799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] PATH_INFO: `/v3/projects/9f5053a1f4e943afb80588bc5869cdf4/users/754119363cde4e28938dd6d623c97e80/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.096329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f5053a1f4e943afb80588bc5869cdf4, user_id=754119363cde4e28938dd6d623c97e80, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:11.099328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.121132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1037/2094] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:11 2022] PUT /identity/v3/projects/9f5053a1f4e943afb80588bc5869cdf4/users/754119363cde4e28938dd6d623c97e80/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:11.133576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb55e6-d2ab-42a4-aec4-6fd23610dd5a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.134652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb55e6-d2ab-42a4-aec4-6fd23610dd5a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.134962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb55e6-d2ab-42a4-aec4-6fd23610dd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.176726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-f4bb55e6-d2ab-42a4-aec4-6fd23610dd5a None None] MFA Rules not processed for user `754119363cde4e28938dd6d623c97e80`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:11.178076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f4bb55e6-d2ab-42a4-aec4-6fd23610dd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:11.248312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:11.261661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1058/2095] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:11.273542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.275479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:11.336635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14357a-a98a-4a67-8c30-cb0660193f8e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:11.340533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:11.341160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.341436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.341734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.351977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:11.353502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.363371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1038/2096] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:11.386750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb55e6-d2ab-42a4-aec4-6fd23610dd5a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.457717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb55e6-d2ab-42a4-aec4-6fd23610dd5a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:11.463183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:11.463841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.464117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.464382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] PATH_INFO: `/v3/users/754119363cde4e28938dd6d623c97e80` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.465590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=754119363cde4e28938dd6d623c97e80)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:11.467586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.533811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754119363cde4e28938dd6d623c97e80'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.545157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.546817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:11.547070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754119363cde4e28938dd6d623c97e80'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.547761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754119363cde4e28938dd6d623c97e80'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.563065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d6294-0426-4c86-a438-c81fbcc6b1a6 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:11.568245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:11.568948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.569230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.569508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] PATH_INFO: `/v3/users/c7705ff2b33048b9b57a6c056d6835d3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.569633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754119363cde4e28938dd6d623c97e80'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.570836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7705ff2b33048b9b57a6c056d6835d3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:11.573106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.594339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e687ad4d-d5e8-4fdb-a236-9fe30a9c00d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754119363cde4e28938dd6d623c97e80'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.604699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1059/2097] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:11 2022] DELETE /identity/v3/users/754119363cde4e28938dd6d623c97e80 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:11.611677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aa008-476f-43dc-b3a3-d4ee1153f0b3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.612715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aa008-476f-43dc-b3a3-d4ee1153f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.613301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aa008-476f-43dc-b3a3-d4ee1153f0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.636066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-789aa008-476f-43dc-b3a3-d4ee1153f0b3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:11.637315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-789aa008-476f-43dc-b3a3-d4ee1153f0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:11.637771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:11.650189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7705ff2b33048b9b57a6c056d6835d3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.665637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7705ff2b33048b9b57a6c056d6835d3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.666390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7705ff2b33048b9b57a6c056d6835d3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.685655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7705ff2b33048b9b57a6c056d6835d3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.713444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7705ff2b33048b9b57a6c056d6835d3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:11.731641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1060/2098] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:11.739244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1039/2099] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:11 2022] DELETE /identity/v3/users/c7705ff2b33048b9b57a6c056d6835d3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:11.745018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.745897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.746204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.746607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.750563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:11.772160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:11.774207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:11.789195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:11.887068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc4780-8394-4597-9686-9b21f82f5ca5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:11.887224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1061/2100] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:11.894052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.895761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:11.896668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:11.897779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.898202 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.898617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.908653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd1de8-ba2a-4bd1-8d39-d49fa68b444a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:11.912968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0fbd-6749-431d-a14f-1c55d3ac0ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:11.913254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:11.913736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0fbd-6749-431d-a14f-1c55d3ac0ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:11.914088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0fbd-6749-431d-a14f-1c55d3ac0ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:11.914479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0fbd-6749-431d-a14f-1c55d3ac0ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:11.915674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.924695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0fbd-6749-431d-a14f-1c55d3ac0ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:11.926115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0fbd-6749-431d-a14f-1c55d3ac0ec1 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:11.928998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1040/2101] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:11.936851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1062/2102] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:11.994033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:11.997356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.022070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba1e102-3a03-4350-98d1-4cff98fce50a service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:12.028304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:12.029346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.029803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.030220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] PATH_INFO: `/v3/users/4b0a532615154eabb8d9ad74838b75dc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.031811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0a532615154eabb8d9ad74838b75dc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:12.034116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:12.110008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0a532615154eabb8d9ad74838b75dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.126522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0a532615154eabb8d9ad74838b75dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.127804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0a532615154eabb8d9ad74838b75dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.137844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0a532615154eabb8d9ad74838b75dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.156923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0a532615154eabb8d9ad74838b75dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.170119 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1041/2103] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:11 2022] DELETE /identity/v3/users/4b0a532615154eabb8d9ad74838b75dc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:12.171971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.172483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.173084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.234240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:12.235447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.317281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:12.318910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.388076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:12.415911 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1063/2104] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:12.428113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:12.430305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.437880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf743d2-a68e-4318-96df-e523131d61e4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:12.456581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:12.457435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.457917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.458339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] PATH_INFO: `/v3/projects/463e6130d4cb463bbef9ebf8dcbe8877` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.461547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=463e6130d4cb463bbef9ebf8dcbe8877)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:12.467028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:12.529134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ccf9a-b4f2-463f-88e1-1bef681572e5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:12.535198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463e6130d4cb463bbef9ebf8dcbe8877', 'request_id': 'req-0fc8e215-ef7b-452b-8b1d-991e55ebf866'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.550503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463e6130d4cb463bbef9ebf8dcbe8877', 'request_id': 'req-0fc8e215-ef7b-452b-8b1d-991e55ebf866'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.558734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463e6130d4cb463bbef9ebf8dcbe8877', 'request_id': 'req-0fc8e215-ef7b-452b-8b1d-991e55ebf866'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.570535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] The token cache is being invalidate because project 463e6130d4cb463bbef9ebf8dcbe8877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:12.570826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.571516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.573299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1042/2105] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:12 2022] DELETE /identity/v3/projects/463e6130d4cb463bbef9ebf8dcbe8877 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:12.576974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:12.578400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.603065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:12.603727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.604070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.604268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.606095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.620616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:12.657503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8e215-ef7b-452b-8b1d-991e55ebf866 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:12.664091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:12.664821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.665101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.665375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] PATH_INFO: `/v3/users/1484f0a2a371428a92feac792a6b8022` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.666537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1484f0a2a371428a92feac792a6b8022)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:12.668571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:12.688807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:12.690424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:12.702662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1064/2106] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:12.704559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:12.706256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.718290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2588d-fa1b-4668-bc74-88b67a3a9ada service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:12.723051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1484f0a2a371428a92feac792a6b8022'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.723218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:12.723838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.724124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.724398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] PATH_INFO: `/v3/projects/9f5053a1f4e943afb80588bc5869cdf4` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.725761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f5053a1f4e943afb80588bc5869cdf4)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:12.727732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:12.741615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1484f0a2a371428a92feac792a6b8022'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.743086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1484f0a2a371428a92feac792a6b8022'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.749382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:12.755291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1484f0a2a371428a92feac792a6b8022'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.774490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4dd7cffc-5b1c-4917-8565-3ee86729fc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1484f0a2a371428a92feac792a6b8022'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.782004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1043/2107] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:12 2022] DELETE /identity/v3/users/1484f0a2a371428a92feac792a6b8022 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:12.790479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f5053a1f4e943afb80588bc5869cdf4', 'request_id': 'req-f8336c9a-6a26-44fa-9f07-9953c35f10a6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.799170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f5053a1f4e943afb80588bc5869cdf4', 'request_id': 'req-f8336c9a-6a26-44fa-9f07-9953c35f10a6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.805438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f5053a1f4e943afb80588bc5869cdf4', 'request_id': 'req-f8336c9a-6a26-44fa-9f07-9953c35f10a6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.816947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] The token cache is being invalidate because project 9f5053a1f4e943afb80588bc5869cdf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:12.817245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.817940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f8336c9a-6a26-44fa-9f07-9953c35f10a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:12.819543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1065/2108] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:12 2022] DELETE /identity/v3/projects/9f5053a1f4e943afb80588bc5869cdf4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:12.824940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc657-4902-4a8e-8188-58a7d4cc2653 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.825256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc657-4902-4a8e-8188-58a7d4cc2653 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.825487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc657-4902-4a8e-8188-58a7d4cc2653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.848705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-efbfc657-4902-4a8e-8188-58a7d4cc2653 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:12.849944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-efbfc657-4902-4a8e-8188-58a7d4cc2653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:12.930658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1044/2109] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:12.934695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d32e-f55a-4102-9324-f64bf3e0c29f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:12.935107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d32e-f55a-4102-9324-f64bf3e0c29f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:12.935869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7d32e-f55a-4102-9324-f64bf3e0c29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:12.980647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2fc7d32e-f55a-4102-9324-f64bf3e0c29f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:12.985008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2fc7d32e-f55a-4102-9324-f64bf3e0c29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.006201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc657-4902-4a8e-8188-58a7d4cc2653 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.007247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1066/2110] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:13.021079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc657-4902-4a8e-8188-58a7d4cc2653 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.028778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.029408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.029678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.029950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.030606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:13.030895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.032572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.052064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1045/2111] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:13.053347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7d32e-f55a-4102-9324-f64bf3e0c29f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.067948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7d32e-f55a-4102-9324-f64bf3e0c29f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.079722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.080360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.080669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.080930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.090384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.092904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.132893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.134813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.136096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1067/2112] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:13.145488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.154760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a16ed-4005-434f-9afe-eb053c587856 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.253231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.253955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.254369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.254830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] PATH_INFO: `/v3/projects/aaa6db43343243a09e8ddc37f8f6c3e7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.256966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa6db43343243a09e8ddc37f8f6c3e7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.259623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.293737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b093c2-2925-4f38-b795-6707d075467a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.307824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.308679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.308957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.309242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.309879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.312047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.323400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:13.330412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa6db43343243a09e8ddc37f8f6c3e7', 'request_id': 'req-b7134060-abcc-4e08-9995-cf2f8f90e9ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.343773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1068/2113] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:13.343773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.343773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.347565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa6db43343243a09e8ddc37f8f6c3e7', 'request_id': 'req-b7134060-abcc-4e08-9995-cf2f8f90e9ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.358394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa6db43343243a09e8ddc37f8f6c3e7', 'request_id': 'req-b7134060-abcc-4e08-9995-cf2f8f90e9ff'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.364388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:13.367644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b04a6-b6f2-4708-a6ea-4e376a28a6be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.388264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] The token cache is being invalidate because project aaa6db43343243a09e8ddc37f8f6c3e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:13.388984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.389978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.391923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1046/2114] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:13 2022] DELETE /identity/v3/projects/aaa6db43343243a09e8ddc37f8f6c3e7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:13.393595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.395947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.446495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.447151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.447435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.447715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] PATH_INFO: `/v3/projects/f9abdf25ff864b10b4c97a195246252a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.449455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9abdf25ff864b10b4c97a195246252a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.451589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.477098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7134060-abcc-4e08-9995-cf2f8f90e9ff admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.482840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3921a-cb01-47ef-881f-9c8ab1d7d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.483540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3921a-cb01-47ef-881f-9c8ab1d7d3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.483730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3921a-cb01-47ef-881f-9c8ab1d7d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.483985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3921a-cb01-47ef-881f-9c8ab1d7d3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.484677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3921a-cb01-47ef-881f-9c8ab1d7d3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.486333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3921a-cb01-47ef-881f-9c8ab1d7d3d3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.500138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1047/2115] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:13.500138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274a6b6-986d-4826-adf2-091c7525fbec None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.500138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274a6b6-986d-4826-adf2-091c7525fbec None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.500138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274a6b6-986d-4826-adf2-091c7525fbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.522946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e274a6b6-986d-4826-adf2-091c7525fbec None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:13.524184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9abdf25ff864b10b4c97a195246252a', 'request_id': 'req-a3337f3d-6481-4623-a700-6988e8e6f829'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.524560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e274a6b6-986d-4826-adf2-091c7525fbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.533968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9abdf25ff864b10b4c97a195246252a', 'request_id': 'req-a3337f3d-6481-4623-a700-6988e8e6f829'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.547184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9abdf25ff864b10b4c97a195246252a', 'request_id': 'req-a3337f3d-6481-4623-a700-6988e8e6f829'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.564051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] The token cache is being invalidate because project f9abdf25ff864b10b4c97a195246252a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:13.565835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.566823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:13.569556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1069/2116] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:13 2022] DELETE /identity/v3/projects/f9abdf25ff864b10b4c97a195246252a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:13.575327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.577522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.610181 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1048/2117] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:13.611905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656b36-a152-4711-a4c5-beeff5b0554b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.612344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656b36-a152-4711-a4c5-beeff5b0554b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.612849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656b36-a152-4711-a4c5-beeff5b0554b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.655713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-85656b36-a152-4711-a4c5-beeff5b0554b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:13.655713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-85656b36-a152-4711-a4c5-beeff5b0554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.665562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1049/2118] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:13.674022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada85f-579d-42c7-9e3d-2a26d142c1e8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.674323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada85f-579d-42c7-9e3d-2a26d142c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.674623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ada85f-579d-42c7-9e3d-2a26d142c1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.681234 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3337f3d-6481-4623-a700-6988e8e6f829 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.686294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44d539-3c01-45b8-97e4-594b37d26dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.686988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44d539-3c01-45b8-97e4-594b37d26dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.687538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44d539-3c01-45b8-97e4-594b37d26dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.687817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44d539-3c01-45b8-97e4-594b37d26dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.688508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44d539-3c01-45b8-97e4-594b37d26dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.690265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44d539-3c01-45b8-97e4-594b37d26dd3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.698046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c2ada85f-579d-42c7-9e3d-2a26d142c1e8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:13.699279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c2ada85f-579d-42c7-9e3d-2a26d142c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.701307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1070/2119] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:13.702656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a9f3-60c4-4aa4-8220-8438622793d2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.703442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a9f3-60c4-4aa4-8220-8438622793d2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.703787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a9f3-60c4-4aa4-8220-8438622793d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.712237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1050/2120] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:13.713952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada85f-579d-42c7-9e3d-2a26d142c1e8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.724741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ada85f-579d-42c7-9e3d-2a26d142c1e8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.725962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1b59a9f3-60c4-4aa4-8220-8438622793d2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:13.727335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1b59a9f3-60c4-4aa4-8220-8438622793d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.732146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.732146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.732146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.732146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] PATH_INFO: `/v3/projects/26e14d6fefb74653bf90ebc5e91751ee/users/b3ebaf358d2e43a3ad646a64111a7f13/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.736705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1071/2121] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:13.740275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a9f3-60c4-4aa4-8220-8438622793d2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.750859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e14d6fefb74653bf90ebc5e91751ee, user_id=b3ebaf358d2e43a3ad646a64111a7f13, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.751448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a9f3-60c4-4aa4-8220-8438622793d2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.755013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.758319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.759242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.759582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.759880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.761493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:13.761793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.763778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.772319 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:13.773397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1051/2122] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:13 2022] PUT /identity/v3/projects/26e14d6fefb74653bf90ebc5e91751ee/users/b3ebaf358d2e43a3ad646a64111a7f13/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:13.774120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.785091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1072/2123] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:13.785091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.796941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b9cca-633a-446f-b623-c8805c7cbbcd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.844078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:13.857543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846233e9-9bc2-4a14-8e48-d262a53086da admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.863094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.863860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.864296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.864694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.865465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0401a5c-b00f-4464-91d9-3f4199c16c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.865611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.866083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0401a5c-b00f-4464-91d9-3f4199c16c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.866362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0401a5c-b00f-4464-91d9-3f4199c16c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.866760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0401a5c-b00f-4464-91d9-3f4199c16c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.867496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0401a5c-b00f-4464-91d9-3f4199c16c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:13.867650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.867862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0401a5c-b00f-4464-91d9-3f4199c16c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.869646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0401a5c-b00f-4464-91d9-3f4199c16c54 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.886203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1052/2124] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:13.886203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeca96-0415-4232-824b-0114134dd119 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.886203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeca96-0415-4232-824b-0114134dd119 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.886203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeca96-0415-4232-824b-0114134dd119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.905382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1073/2125] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:13.906578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.916144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fdfeca96-0415-4232-824b-0114134dd119 None None] MFA Rules not processed for user `b3ebaf358d2e43a3ad646a64111a7f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:13.917407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fdfeca96-0415-4232-824b-0114134dd119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:13.973497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1012fe8-d856-4245-8887-1f82000369d9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:13.978847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:13.979442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:13.979722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:13.979993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:13.980666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:13.982669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:13.986603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1053/2126] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:13.987437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfeca96-0415-4232-824b-0114134dd119 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:13.998318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfeca96-0415-4232-824b-0114134dd119 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.004213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.004924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.005192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.005478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.006400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.008038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.015450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1074/2127] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:14.015450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.017238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:14.033927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1054/2128] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:13 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:14.033927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.041664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37920189-2d00-4ef3-95f6-2f1a2f8bff8d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.080205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46abea-6404-42bd-ab95-7d56a7fa0aef admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.084505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.085481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.085762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.086329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.101043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.101581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.101860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.102169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.102810 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.104793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.107855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:14.128906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1055/2129] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:14 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:14.129738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.142303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d46302-63c6-458e-b374-6c9f867a0b28 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.147566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.149202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.149523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.150168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.150445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.150715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.151374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.153050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.159178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1075/2130] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:14.159998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.164684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1056/2131] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:14.168952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.171818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171398e-cb30-4f89-9496-d80c4bb6b5f6 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.177808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.178484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.178772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.179037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.179351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027a39f-2747-4b36-b92e-2d6ecf63e4d3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.179723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.181438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.184687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.185089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.185491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.185676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.186433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.188150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.193213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1076/2132] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:14.197672 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.204027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1057/2133] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:14.208286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.208832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27afda-4bf1-4d24-ad3d-2d2e5815c778 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.214299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.214912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.215192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.215457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.216107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.218048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.222338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51696cb-7d8e-4f6c-9dea-9404ada2e65b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.228221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1077/2134] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:14.228739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb9e84-0d46-4ae3-9777-b07fcbffb552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.229652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb9e84-0d46-4ae3-9777-b07fcbffb552 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.229928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb9e84-0d46-4ae3-9777-b07fcbffb552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.230195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb9e84-0d46-4ae3-9777-b07fcbffb552 admin admin] PATH_INFO: `/v3/projects/b29463eb62214f8f9cd949e8526bafc2/users/1df00ac900d142b995810155e6049921/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.232230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.244623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d14f8f-6701-486f-a39e-bec011aeb635 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.250485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae3e20-107f-4dcf-b389-6f188afe3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.251162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae3e20-107f-4dcf-b389-6f188afe3c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.251290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb9e84-0d46-4ae3-9777-b07fcbffb552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29463eb62214f8f9cd949e8526bafc2, user_id=1df00ac900d142b995810155e6049921, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.251454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae3e20-107f-4dcf-b389-6f188afe3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.251714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae3e20-107f-4dcf-b389-6f188afe3c7d admin admin] PATH_INFO: `/v3/projects/f350884ae98c454aac0c1e6750d06bcb/users/373b7e558c464082ab451c5dd6b361cc/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.254872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb9e84-0d46-4ae3-9777-b07fcbffb552 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.272039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1058/2135] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:14 2022] PUT /identity/v3/projects/b29463eb62214f8f9cd949e8526bafc2/users/1df00ac900d142b995810155e6049921/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:14.272282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae3e20-107f-4dcf-b389-6f188afe3c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f350884ae98c454aac0c1e6750d06bcb, user_id=373b7e558c464082ab451c5dd6b361cc, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.275043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae3e20-107f-4dcf-b389-6f188afe3c7d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.284756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b6f5-a537-4413-a313-5c1675e6b8e5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.285049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b6f5-a537-4413-a313-5c1675e6b8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.285327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b6f5-a537-4413-a313-5c1675e6b8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.293531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:14.306527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1078/2136] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:14 2022] PUT /identity/v3/projects/f350884ae98c454aac0c1e6750d06bcb/users/373b7e558c464082ab451c5dd6b361cc/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:14.311688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.312258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.312555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.322068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2b30b6f5-a537-4413-a313-5c1675e6b8e5 None None] MFA Rules not processed for user `1df00ac900d142b995810155e6049921`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:14.323318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2b30b6f5-a537-4413-a313-5c1675e6b8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:14.334458 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:14.335826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:14.408045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1059/2137] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:14.413638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30b6f5-a537-4413-a313-5c1675e6b8e5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.416452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1079/2138] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:14.427264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.429033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:14.431686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30b6f5-a537-4413-a313-5c1675e6b8e5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.438189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.439321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.439891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.440166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.440874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.443354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.485561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1060/2139] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:14 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:14.489379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.528885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ec1b-9752-4ab3-bb3e-0079c9c7882a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.566209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:14.584401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f47cf9-e09c-4503-93e6-5b13810d0617 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.589952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.590583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.590866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.591134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.591797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.594295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.604076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.605376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.605670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.605948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.618286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.620364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.625220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1061/2140] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:14 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:14.630732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.635349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1080/2141] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:03:14.645362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f57543-b5b9-48fb-a846-85f375b8471b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.653609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.654216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.654560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.654901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.655958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.657455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.673880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1062/2142] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:14.678884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.696694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3654a1f-37c8-47bd-a551-359fd7833f35 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.703810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.704566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.705009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.705284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] PATH_INFO: `/v3/projects/6f04ad4ff11d4b2c9088b6d5532d8749/users/c9e5f74583a8498cbc17c3ecd1252347/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.723345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f04ad4ff11d4b2c9088b6d5532d8749, user_id=c9e5f74583a8498cbc17c3ecd1252347, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.725854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.751141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1081/2143] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:14 2022] PUT /identity/v3/projects/6f04ad4ff11d4b2c9088b6d5532d8749/users/c9e5f74583a8498cbc17c3ecd1252347/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:14.757420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.769739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad52e9e-1f40-4c78-9540-ae65257f63e4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.837381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:14.856459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13ef85-f08c-432e-a64e-6363f5ac6124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.857436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13ef85-f08c-432e-a64e-6363f5ac6124 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.857859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13ef85-f08c-432e-a64e-6363f5ac6124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.858296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13ef85-f08c-432e-a64e-6363f5ac6124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.859214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13ef85-f08c-432e-a64e-6363f5ac6124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.861122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13ef85-f08c-432e-a64e-6363f5ac6124 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.874500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1063/2144] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:14.879646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:14.892217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af13fa7-3ed3-4c8a-8db1-94c662347073 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:14.898345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:14.899170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.899563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.899907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] PATH_INFO: `/v3/projects/6f04ad4ff11d4b2c9088b6d5532d8749/users/c9e5f74583a8498cbc17c3ecd1252347/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.902188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f04ad4ff11d4b2c9088b6d5532d8749, user_id=c9e5f74583a8498cbc17c3ecd1252347, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:14.905061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:14.918010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1082/2145] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:14 2022] PUT /identity/v3/projects/6f04ad4ff11d4b2c9088b6d5532d8749/users/c9e5f74583a8498cbc17c3ecd1252347/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:14.929518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd80a8-dea3-49c6-8717-21afd0d978c3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:14.929843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd80a8-dea3-49c6-8717-21afd0d978c3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:14.930310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd80a8-dea3-49c6-8717-21afd0d978c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:14.964208 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3dbd80a8-dea3-49c6-8717-21afd0d978c3 None None] MFA Rules not processed for user `c9e5f74583a8498cbc17c3ecd1252347`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:14.970256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3dbd80a8-dea3-49c6-8717-21afd0d978c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:15.058563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1064/2146] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:15.070341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:15.072133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:15.138535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294166-44ef-4012-a0dc-70d19bc6623c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:15.142494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:15.143122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:15.143440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:15.143709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:15.210172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:15.211654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:15.222749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1083/2147] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:16.070250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4f377-93be-4157-b2e0-6b523907a086 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:16.070987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4f377-93be-4157-b2e0-6b523907a086 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:16.070987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4f377-93be-4157-b2e0-6b523907a086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:16.151698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5ce4f377-93be-4157-b2e0-6b523907a086 None None] MFA Rules not processed for user `373b7e558c464082ab451c5dd6b361cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:16.152867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5ce4f377-93be-4157-b2e0-6b523907a086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:16.333356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1065/2148] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:16.343098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:16.480268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:16.507272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6e733-5440-4899-92d4-b9a4fe7291d6 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:16.525142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:16.528038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:16.528700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:16.529521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:16.532462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:16.540977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:16.612940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4f377-93be-4157-b2e0-6b523907a086 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:16.618836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1084/2149] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:16 2022] POST /identity/v3/projects => generated 371 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:16.628267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4f377-93be-4157-b2e0-6b523907a086 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:16.786808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:16.788457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:16.788884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:16.790125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:16.790972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:16.793759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:16.804408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:16.827364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1066/2150] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:16 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:16.832272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:16.843091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ca398-9c65-4e7f-8ceb-892dd1f6fe8e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:16.849097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:16.851673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:16.852225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:16.852584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:16.853680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:16.855232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:16.867606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1085/2151] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:16.872457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:16.886901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09875a1-cc0d-4b9f-9a13-e5af4e1a7630 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:16.895354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:16.896200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:16.896512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:16.896857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] PATH_INFO: `/v3/projects/398d9e1e85a9420f850746df7f796910/users/93a04ec0d9894df582743ea36bd8fede/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:16.915783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=398d9e1e85a9420f850746df7f796910, user_id=93a04ec0d9894df582743ea36bd8fede, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:16.919004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:16.937313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1067/2152] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:16 2022] PUT /identity/v3/projects/398d9e1e85a9420f850746df7f796910/users/93a04ec0d9894df582743ea36bd8fede/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:16.948268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:16.961000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6d998-4569-4a45-bcc5-8d2115901081 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:17.029124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:17.030167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:17.030167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:17.030306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:17.030934 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:17.032639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:17.043491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1086/2153] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:17.047750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:17.064594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d65c4b-90b7-4d84-9b3a-e2d710e3a7c9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:17.070493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:17.071413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:17.071536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:17.071945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] PATH_INFO: `/v3/projects/398d9e1e85a9420f850746df7f796910/users/93a04ec0d9894df582743ea36bd8fede/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:17.073893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=398d9e1e85a9420f850746df7f796910, user_id=93a04ec0d9894df582743ea36bd8fede, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:17.084737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:17.119705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1068/2154] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:17 2022] PUT /identity/v3/projects/398d9e1e85a9420f850746df7f796910/users/93a04ec0d9894df582743ea36bd8fede/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:17.859976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:17.873858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f15211-e228-4406-a9c9-d8a9a1631276 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:17.941459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:17.942286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:17.942556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:17.942824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] PATH_INFO: `/v3/users/b3ebaf358d2e43a3ad646a64111a7f13` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:17.944026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ebaf358d2e43a3ad646a64111a7f13)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:17.946018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:17.993613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:18.012317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eb475-51dd-4e13-8aba-f0335be57c00 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:18.012531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ebaf358d2e43a3ad646a64111a7f13'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.027963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:18.029026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:18.029197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ebaf358d2e43a3ad646a64111a7f13'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.029600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:18.029943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] PATH_INFO: `/v3/users/1df00ac900d142b995810155e6049921` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:18.030467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ebaf358d2e43a3ad646a64111a7f13'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.031315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df00ac900d142b995810155e6049921)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:18.033389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:18.042853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ebaf358d2e43a3ad646a64111a7f13'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.045966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:18.087741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0749f720-dd4b-4c23-b085-65163f8abaed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ebaf358d2e43a3ad646a64111a7f13'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.102984 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1087/2155] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:17 2022] DELETE /identity/v3/users/b3ebaf358d2e43a3ad646a64111a7f13 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:18.105093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:18.105601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:18.105981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:18.122105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df00ac900d142b995810155e6049921'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.135819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df00ac900d142b995810155e6049921'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.137367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df00ac900d142b995810155e6049921'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.147078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:18.148468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:18.150567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df00ac900d142b995810155e6049921'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.172603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5755796a-e52e-4758-b401-638e0216dc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df00ac900d142b995810155e6049921'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:18.186115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1069/2156] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:17 2022] DELETE /identity/v3/users/1df00ac900d142b995810155e6049921 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:18.188194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d179aa-290c-4dc2-92ba-4752b0dd7bc1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:18.188681 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d179aa-290c-4dc2-92ba-4752b0dd7bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:18.189158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d179aa-290c-4dc2-92ba-4752b0dd7bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:18.218658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-44d179aa-290c-4dc2-92ba-4752b0dd7bc1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:18.221048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-44d179aa-290c-4dc2-92ba-4752b0dd7bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:18.280570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1088/2157] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:18.292595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:18.294320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:18.369047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1070/2158] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:18.414259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a89021-34df-4ae8-8afb-f74a045d21da None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:18.419113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:18.419659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:18.419936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:18.477375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] MFA Rules not processed for user `93a04ec0d9894df582743ea36bd8fede`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:18.478782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:18.499112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:18.499872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:18.500501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:18.500844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:18.508854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=6,keystone:SELECT=84,keystone:DELETE=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:18.520747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:18.526605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:18.542878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1089/2159] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:18.544656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:18.554876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611533d-2dec-4d40-8e21-83c80d66cfeb service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:18.572002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:18.572002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:18.576851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:18.578231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:18.594178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:18.595850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:18.605007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1071/2160] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:18.610513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1090/2161] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:18.621407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:18.623136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:18.689577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ff5f8-6438-4433-9b23-b974128c0f0c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:18.699944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:18.700623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:18.700900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:18.701168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:18.702977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:18.790988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:18.792752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:18.803926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1072/2162] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:19.393859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:19.394932 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:19.415538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d08ab9-93ae-4d04-981e-56a141897569 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:19.430334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:19.431241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:19.431530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:19.431803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] PATH_INFO: `/v3/projects/b29463eb62214f8f9cd949e8526bafc2` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:19.433954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29463eb62214f8f9cd949e8526bafc2)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:19.436569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:19.484838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:19.530386 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29463eb62214f8f9cd949e8526bafc2', 'request_id': 'req-4421d4f7-6b3c-46d3-a986-1f64328846d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.549138 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29463eb62214f8f9cd949e8526bafc2', 'request_id': 'req-4421d4f7-6b3c-46d3-a986-1f64328846d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.561749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29463eb62214f8f9cd949e8526bafc2', 'request_id': 'req-4421d4f7-6b3c-46d3-a986-1f64328846d7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.569987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:19.571309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:19.589018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] The token cache is being invalidate because project b29463eb62214f8f9cd949e8526bafc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:19.589018 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.589572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f632db-e08d-493d-8281-d6ae59bba5a5 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:19.595377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.597482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1091/2163] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:19 2022] DELETE /identity/v3/projects/b29463eb62214f8f9cd949e8526bafc2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:19.604223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:19.607955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:19.677428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:19.679232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:19.679577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:19.679964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] PATH_INFO: `/v3/projects/26e14d6fefb74653bf90ebc5e91751ee` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:19.684807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e14d6fefb74653bf90ebc5e91751ee)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:19.688282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:19.695272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:19.699882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421d4f7-6b3c-46d3-a986-1f64328846d7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:19.704764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:19.705651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:19.705651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:19.705935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] PATH_INFO: `/v3/users/c9e5f74583a8498cbc17c3ecd1252347` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:19.707146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e5f74583a8498cbc17c3ecd1252347)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:19.710060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:19.761762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e14d6fefb74653bf90ebc5e91751ee', 'request_id': 'req-c89a8844-3ba1-499d-a573-1d775a5bc4e2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.773425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e14d6fefb74653bf90ebc5e91751ee', 'request_id': 'req-c89a8844-3ba1-499d-a573-1d775a5bc4e2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.783719 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e14d6fefb74653bf90ebc5e91751ee', 'request_id': 'req-c89a8844-3ba1-499d-a573-1d775a5bc4e2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.799878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] The token cache is being invalidate because project 26e14d6fefb74653bf90ebc5e91751ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:19.800594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.801401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c89a8844-3ba1-499d-a573-1d775a5bc4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.803394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e5f74583a8498cbc17c3ecd1252347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.803619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1073/2164] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:19 2022] DELETE /identity/v3/projects/26e14d6fefb74653bf90ebc5e91751ee => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:19.808979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0808706-c096-4312-acca-0cf1f7f0bb3c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:19.809920 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0808706-c096-4312-acca-0cf1f7f0bb3c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:19.810428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0808706-c096-4312-acca-0cf1f7f0bb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:19.831084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e5f74583a8498cbc17c3ecd1252347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.832993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e5f74583a8498cbc17c3ecd1252347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.856122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b0808706-c096-4312-acca-0cf1f7f0bb3c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:19.857007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e5f74583a8498cbc17c3ecd1252347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.857448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b0808706-c096-4312-acca-0cf1f7f0bb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:19.890247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6d6edb88-80e6-47d7-be7a-9098edba8d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e5f74583a8498cbc17c3ecd1252347'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:19.901830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1092/2165] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:19 2022] DELETE /identity/v3/users/c9e5f74583a8498cbc17c3ecd1252347 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:19.984169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1074/2166] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:19.992184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35ee6-a796-45e4-acc7-9803d72ab727 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:19.992505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35ee6-a796-45e4-acc7-9803d72ab727 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:19.992861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35ee6-a796-45e4-acc7-9803d72ab727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.047726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cef35ee6-a796-45e4-acc7-9803d72ab727 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:20.054621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cef35ee6-a796-45e4-acc7-9803d72ab727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:20.110877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:20.154342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1093/2167] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:20.158827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0808706-c096-4312-acca-0cf1f7f0bb3c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.176966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0808706-c096-4312-acca-0cf1f7f0bb3c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.182468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.183094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.183365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.183632 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.184307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:20.184638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.186807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.205899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1075/2168] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:03:20.210906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef35ee6-a796-45e4-acc7-9803d72ab727 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.227755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef35ee6-a796-45e4-acc7-9803d72ab727 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.235297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.235996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.236272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.236621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.237300 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.239332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.301408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1094/2169] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:20 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:20.306092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.318538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af85ef-1105-406e-8fc0-c31d95e5aece admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.408187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.408869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.409149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.409420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.410848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.424502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.441186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.443420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:20.472183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1076/2170] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:20 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:20.476981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.499531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facff9e6-2ea0-487b-ad1f-e13305ae3d09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.509674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb413093-3c4c-4e3e-b5ba-6086069e8681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.513111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb413093-3c4c-4e3e-b5ba-6086069e8681 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.514044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb413093-3c4c-4e3e-b5ba-6086069e8681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.514855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb413093-3c4c-4e3e-b5ba-6086069e8681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.515931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb413093-3c4c-4e3e-b5ba-6086069e8681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.520453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb413093-3c4c-4e3e-b5ba-6086069e8681 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.532271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:20.546952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1077/2171] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:20.546952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93034c6-0ccf-4e1f-aca9-8c8df76d2bdb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.546952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93034c6-0ccf-4e1f-aca9-8c8df76d2bdb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.547532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93034c6-0ccf-4e1f-aca9-8c8df76d2bdb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.549083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1078/2172] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:03:20.557255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93034c6-0ccf-4e1f-aca9-8c8df76d2bdb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.576576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2e0e9-2d2e-4ad3-86be-ffe36eed37e7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.582490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.582808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93034c6-0ccf-4e1f-aca9-8c8df76d2bdb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.589235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be539e-e705-4279-a952-52af3f94e3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.590284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be539e-e705-4279-a952-52af3f94e3de admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.590735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be539e-e705-4279-a952-52af3f94e3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.591239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be539e-e705-4279-a952-52af3f94e3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.592354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be539e-e705-4279-a952-52af3f94e3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.594521 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be539e-e705-4279-a952-52af3f94e3de admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.595240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.596066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.596348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.604567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:20.620408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1079/2173] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:20.622168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.622899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.623178 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.652468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:20.652468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:20.702987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.704620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.714831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1095/2174] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:03:20.715654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.725774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc93756-4459-459b-82a4-e72cf4787201 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.731215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.731890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.732189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.732462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] PATH_INFO: `/v3/projects/5eafd54351bd4049b87ad308a3638f80/users/2ab9930d6ce54cacbb3d4fd6313294d5/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.752683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1080/2175] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:20.753597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.756165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:20.760923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eafd54351bd4049b87ad308a3638f80, user_id=2ab9930d6ce54cacbb3d4fd6313294d5, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.763683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.777386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca28bc2-4a39-41ee-babb-35dcd0544fad None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.782639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1096/2176] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:20 2022] PUT /identity/v3/projects/5eafd54351bd4049b87ad308a3638f80/users/2ab9930d6ce54cacbb3d4fd6313294d5/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:20.783661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.785338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:20.834164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:20.854441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.855209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.855551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.855913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] PATH_INFO: `/v3/users/51db5ef052144879b53979268ac4ceb5` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.857206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] RBAC: Authorizing `identity:delete_user(user_id=51db5ef052144879b53979268ac4ceb5)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.859307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.864667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43922d42-9eae-4675-9fc5-ab1db395e24b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:20.870023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:20.871094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.871373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.871644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] PATH_INFO: `/v3/projects/6f04ad4ff11d4b2c9088b6d5532d8749` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.872820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f04ad4ff11d4b2c9088b6d5532d8749)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:20.876382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:20.936315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51db5ef052144879b53979268ac4ceb5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.946020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51db5ef052144879b53979268ac4ceb5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.946705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51db5ef052144879b53979268ac4ceb5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.951026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f04ad4ff11d4b2c9088b6d5532d8749', 'request_id': 'req-457f19f1-22c4-4864-af46-e9fe5a16e390'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.956684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51db5ef052144879b53979268ac4ceb5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.961153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f04ad4ff11d4b2c9088b6d5532d8749', 'request_id': 'req-457f19f1-22c4-4864-af46-e9fe5a16e390'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.968864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f04ad4ff11d4b2c9088b6d5532d8749', 'request_id': 'req-457f19f1-22c4-4864-af46-e9fe5a16e390'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.974535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51db5ef052144879b53979268ac4ceb5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.981787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] The token cache is being invalidate because project 6f04ad4ff11d4b2c9088b6d5532d8749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:20.981940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1081/2177] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:20 2022] DELETE /identity/v3/users/51db5ef052144879b53979268ac4ceb5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:20.982173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.982992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-457f19f1-22c4-4864-af46-e9fe5a16e390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:20.983098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:20.984757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1097/2178] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:20 2022] DELETE /identity/v3/projects/6f04ad4ff11d4b2c9088b6d5532d8749 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:20.984865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:20.985752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5314b6-67eb-4eb5-8ff6-26d000c0d584 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:20.986042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5314b6-67eb-4eb5-8ff6-26d000c0d584 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:20.986307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5314b6-67eb-4eb5-8ff6-26d000c0d584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:20.997572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:21.009723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6a5314b6-67eb-4eb5-8ff6-26d000c0d584 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:21.011747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6a5314b6-67eb-4eb5-8ff6-26d000c0d584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.076245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc2bd5-da32-4d88-b78b-a788f9e0e37c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.082156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.083136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.083479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.083867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.086018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.104402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1098/2179] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:21.106710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bb4e1-ec58-43e6-ab0a-069b098087ee None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.106985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bb4e1-ec58-43e6-ab0a-069b098087ee None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.107295 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bb4e1-ec58-43e6-ab0a-069b098087ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.110375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.113281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.125914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1082/2180] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:21.127060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.128965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.132177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-719bb4e1-ec58-43e6-ab0a-069b098087ee None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:21.133523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-719bb4e1-ec58-43e6-ab0a-069b098087ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.144624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1099/2181] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:21.153181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20c8b1-dc15-4bec-9dec-dc33f0000386 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.153570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20c8b1-dc15-4bec-9dec-dc33f0000386 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.153959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20c8b1-dc15-4bec-9dec-dc33f0000386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.179309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1d20c8b1-dc15-4bec-9dec-dc33f0000386 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:21.180891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1d20c8b1-dc15-4bec-9dec-dc33f0000386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.190406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1100/2182] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:21.196897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20c8b1-dc15-4bec-9dec-dc33f0000386 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.206597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20c8b1-dc15-4bec-9dec-dc33f0000386 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.213044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.213831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.214140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.214480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.215422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:21.215811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.217641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.224342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d750e4f-974f-4fde-84b2-57a9d6234e3f service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.230876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.235487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.235487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.235487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.239348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1101/2183] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:21.240855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.243527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.245562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.247086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.257487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e053c39-126a-4b1d-b381-ef9326e4042b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.260709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1083/2184] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:21.262666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.264200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.265100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.265591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.265928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] PATH_INFO: `/v3/projects/66136f123f5b4ad0a824920bbdf9d245` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.267183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66136f123f5b4ad0a824920bbdf9d245)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.269247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.278698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d0f45-16a7-4f94-b42a-ae44f30daf06 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.285527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.286150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.286512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.286833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.287618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.290371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.321972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:21.324149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1084/2185] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:21.328633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.342713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66136f123f5b4ad0a824920bbdf9d245', 'request_id': 'req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:21.360634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66136f123f5b4ad0a824920bbdf9d245', 'request_id': 'req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:21.368481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66136f123f5b4ad0a824920bbdf9d245', 'request_id': 'req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:21.383536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] The token cache is being invalidate because project 66136f123f5b4ad0a824920bbdf9d245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:21.383852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:21.384574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6e1ceb7b-c055-4d0c-8875-faa24019ccc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:21.386163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1102/2186] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:21 2022] DELETE /identity/v3/projects/66136f123f5b4ad0a824920bbdf9d245 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:21.393389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbf6dc-30c2-4568-8c73-642c92860209 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.395370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbf6dc-30c2-4568-8c73-642c92860209 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.395805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbf6dc-30c2-4568-8c73-642c92860209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.416536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb59979-7fe1-4184-93b6-7c4c53990a23 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.423249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.424066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.425003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.425362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.426591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.427656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-13fbf6dc-30c2-4568-8c73-642c92860209 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:21.429602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13fbf6dc-30c2-4568-8c73-642c92860209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.429891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.445271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1103/2187] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:21.453573 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e987a-f30c-4bdb-81ea-3e7f213f2c6d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.453907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e987a-f30c-4bdb-81ea-3e7f213f2c6d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.454330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e987a-f30c-4bdb-81ea-3e7f213f2c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.461032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1085/2188] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:21.465362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.467731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.475700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-213e987a-f30c-4bdb-81ea-3e7f213f2c6d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:21.477302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-213e987a-f30c-4bdb-81ea-3e7f213f2c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.478404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:21.483191 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b31a-5b91-4e79-a3cd-96b5f4ccda46 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.489991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.491162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.492474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.492608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1104/2189] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:21.492760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.492760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.494372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.495679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e987a-f30c-4bdb-81ea-3e7f213f2c6d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.506861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e987a-f30c-4bdb-81ea-3e7f213f2c6d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.511573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1086/2190] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:21.512697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.513227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.513849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.514133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.514425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.515094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:21.515400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.517066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.525807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139001b-cf75-41e4-bd32-8b6628753d6e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.530363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1105/2191] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:03:21.536062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.538836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.539462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.539783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.540055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.540790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.542858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.547164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e193e01-7663-4a6d-a066-aa7fc598acb8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.552228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.552878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.553160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.553432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.554075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.559007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.559743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1087/2192] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:21.560833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.581943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c797e97-19ce-4676-905a-20cb14a479de admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.590012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ae82-5cb5-40e9-8ef4-f33bd1377d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.591182 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ae82-5cb5-40e9-8ef4-f33bd1377d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.591603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ae82-5cb5-40e9-8ef4-f33bd1377d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.592014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ae82-5cb5-40e9-8ef4-f33bd1377d1e admin admin] PATH_INFO: `/v3/projects/51bcb82c807b4a22a80a2bb9df888c5a/users/ae2edac9b70e4f83ab6734479d359f7f/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.603651 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1106/2193] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:21.606233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.627013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ae82-5cb5-40e9-8ef4-f33bd1377d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bcb82c807b4a22a80a2bb9df888c5a, user_id=ae2edac9b70e4f83ab6734479d359f7f, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.631984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ae82-5cb5-40e9-8ef4-f33bd1377d1e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.653765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1088/2194] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:21 2022] PUT /identity/v3/projects/51bcb82c807b4a22a80a2bb9df888c5a/users/ae2edac9b70e4f83ab6734479d359f7f/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:21.668712 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dd07a-2f46-4787-9291-b0088641a71c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.668989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dd07a-2f46-4787-9291-b0088641a71c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.669262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dd07a-2f46-4787-9291-b0088641a71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.712936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-963dd07a-2f46-4787-9291-b0088641a71c None None] MFA Rules not processed for user `ae2edac9b70e4f83ab6734479d359f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:21.714289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-963dd07a-2f46-4787-9291-b0088641a71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:21.744271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834b38c-4fd9-4b67-8e62-e8ce4740d289 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.801579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1089/2195] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:21.806065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dd07a-2f46-4787-9291-b0088641a71c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.816544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.817222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.817531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.817802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.818477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.820415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.849677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1107/2196] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:21.853421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.869673 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644fc5f7-8aa9-441a-a310-0f1d43367f6f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.882243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.882877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.883162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.883482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.884137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.885779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.892052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:21.898118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1108/2197] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:21.905177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.915549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dd07a-2f46-4787-9291-b0088641a71c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.918313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cae2b9-ca4e-4d96-9dab-2b0f1d9048f5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.920914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.921560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.921813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.922085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.922749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.923720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.924712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.924994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.925303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.925578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.926258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.927864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:21.944989 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1109/2198] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:21.946383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.958784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37907c4a-949b-4f07-a2c4-2171a4e9f686 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:21.963312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1090/2199] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:21.963622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c521a13-ed7c-44a0-a3f6-0340cec36128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:21.964379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c521a13-ed7c-44a0-a3f6-0340cec36128 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:21.964823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c521a13-ed7c-44a0-a3f6-0340cec36128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:21.965123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c521a13-ed7c-44a0-a3f6-0340cec36128 admin admin] PATH_INFO: `/v3/projects/d8dea26ee3b04f11a491c5d2fc843350/users/b03e7a8ab6c34868af811ade7426e09c/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:21.968783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:21.994059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c521a13-ed7c-44a0-a3f6-0340cec36128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dea26ee3b04f11a491c5d2fc843350, user_id=b03e7a8ab6c34868af811ade7426e09c, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:21.997447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c521a13-ed7c-44a0-a3f6-0340cec36128 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:22.022398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:22.032917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1110/2200] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:21 2022] PUT /identity/v3/projects/d8dea26ee3b04f11a491c5d2fc843350/users/b03e7a8ab6c34868af811ade7426e09c/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:22.046825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.047232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.047774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.076687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8cc18-edcf-4bbe-9dca-cdf266744ad2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:22.093742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] MFA Rules not processed for user `b03e7a8ab6c34868af811ade7426e09c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:22.096724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:22.201653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1111/2201] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:22.213141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:22.214775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:22.216990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:22.217822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.218175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.218517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.219522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:22.223067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:22.266267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1091/2202] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:21 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:22.273673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:22.302760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b260498-419b-4b18-a1be-962724b55761 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:22.308853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:22.309526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.309797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.310069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.310768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:22.312472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:22.328464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1092/2203] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:22.333558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:22.345895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89246e3-c694-4eef-aff0-a7e0fa107093 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:22.357314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:22.360299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.360847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.361642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.372480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82d20-8c77-4877-91d2-6d3bdf401222 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:22.381574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:22.384687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:22.387483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa4113-0045-48e8-8ac8-af3d858221a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:22.388281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4113-0045-48e8-8ac8-af3d858221a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.388691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4113-0045-48e8-8ac8-af3d858221a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.389113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa4113-0045-48e8-8ac8-af3d858221a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.390026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa4113-0045-48e8-8ac8-af3d858221a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:22.392482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa4113-0045-48e8-8ac8-af3d858221a7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:22.406121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1112/2204] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:22.415630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1093/2205] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:22.425846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:22.439528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e2fef-5883-4c6f-aff2-d3cc98459231 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:22.445617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:22.446389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.446618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.446892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] PATH_INFO: `/v3/projects/3d8f2b51eda34f3f8aecc84e9c770300/users/465fc5f283bb43ab9581405e2ed22156/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.502729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8f2b51eda34f3f8aecc84e9c770300, user_id=465fc5f283bb43ab9581405e2ed22156, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:22.508195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:22.539478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1113/2206] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:22 2022] PUT /identity/v3/projects/3d8f2b51eda34f3f8aecc84e9c770300/users/465fc5f283bb43ab9581405e2ed22156/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:22.551314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1059dd6-cbe3-4e22-8cb5-b106d404c7b8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.552262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1059dd6-cbe3-4e22-8cb5-b106d404c7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.552922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1059dd6-cbe3-4e22-8cb5-b106d404c7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.617363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c1059dd6-cbe3-4e22-8cb5-b106d404c7b8 None None] MFA Rules not processed for user `465fc5f283bb43ab9581405e2ed22156`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:22.619442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c1059dd6-cbe3-4e22-8cb5-b106d404c7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:22.730269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1094/2207] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:22.736552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:22.764861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:22.890496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea082b-46ce-4bae-a2f5-a574ffc54014 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:22.906324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:22.906966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:22.907245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:22.907513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:22.908318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:22.910586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:22.991724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1114/2208] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:22 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:22.999756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1059dd6-cbe3-4e22-8cb5-b106d404c7b8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:23.032810 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1059dd6-cbe3-4e22-8cb5-b106d404c7b8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:23.129907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:23.176795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:23.179718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:23.180328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:23.182179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:23.183849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:23.190605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:23.228030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1095/2209] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:22 2022] POST /identity/v3/users => generated 299 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:23.233572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:23.252129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401facd-2172-4059-a37f-3cdb8b07473b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:23.260787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:23.261825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:23.262267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:23.262736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:23.263842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:23.266517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:23.280772 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1115/2210] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:23.285288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:23.297548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd54da-e9b4-4448-9a83-828fb5523e31 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:23.302614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:23.303321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:23.303628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:23.303906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] PATH_INFO: `/v3/projects/6778af1293444c37b5b0c9c0813f0950/users/60eac013f2494c3fb6f4c3d21fcdfd8d/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:23.331295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6778af1293444c37b5b0c9c0813f0950, user_id=60eac013f2494c3fb6f4c3d21fcdfd8d, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:23.336687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:23.363995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1096/2211] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:23 2022] PUT /identity/v3/projects/6778af1293444c37b5b0c9c0813f0950/users/60eac013f2494c3fb6f4c3d21fcdfd8d/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:23.365265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:23.379851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5bfd4-718a-4dd6-8131-2176114722fd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:23.460310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf61f7c-619f-4307-a2c9-96e6cbbd673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:23.461073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf61f7c-619f-4307-a2c9-96e6cbbd673f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:23.461674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf61f7c-619f-4307-a2c9-96e6cbbd673f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:23.461674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf61f7c-619f-4307-a2c9-96e6cbbd673f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:23.462255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf61f7c-619f-4307-a2c9-96e6cbbd673f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:23.463963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf61f7c-619f-4307-a2c9-96e6cbbd673f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:23.477235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1116/2212] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:23.481098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:23.494547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881616f6-4c06-4190-ad33-56d043ef7754 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:23.499790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:23.503474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:23.503474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:23.503708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] PATH_INFO: `/v3/projects/6778af1293444c37b5b0c9c0813f0950/users/60eac013f2494c3fb6f4c3d21fcdfd8d/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:23.506972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6778af1293444c37b5b0c9c0813f0950, user_id=60eac013f2494c3fb6f4c3d21fcdfd8d, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:23.511470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:23.527101 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1097/2213] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:23 2022] PUT /identity/v3/projects/6778af1293444c37b5b0c9c0813f0950/users/60eac013f2494c3fb6f4c3d21fcdfd8d/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:23.537371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e5f83-1c75-4518-a199-32d35945fba8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:23.538089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e5f83-1c75-4518-a199-32d35945fba8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:23.538548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e5f83-1c75-4518-a199-32d35945fba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:23.586642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6f0e5f83-1c75-4518-a199-32d35945fba8 None None] MFA Rules not processed for user `60eac013f2494c3fb6f4c3d21fcdfd8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:23.588258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6f0e5f83-1c75-4518-a199-32d35945fba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:23.715601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1117/2214] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:23.729100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:23.735296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:23.833370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d42347-59e8-4817-b915-765582f4fdba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:23.847588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baa949-6e87-453a-bbe0-7b232db5d1e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:23.848302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa949-6e87-453a-bbe0-7b232db5d1e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:23.848689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa949-6e87-453a-bbe0-7b232db5d1e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:23.849324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa949-6e87-453a-bbe0-7b232db5d1e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:23.981733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baa949-6e87-453a-bbe0-7b232db5d1e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:23.985526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baa949-6e87-453a-bbe0-7b232db5d1e2 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:24.002775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1098/2215] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:25.850156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e5f83-1c75-4518-a199-32d35945fba8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:25.861766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e5f83-1c75-4518-a199-32d35945fba8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:25.866625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:25.867436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:25.868170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:25.868600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:25.879733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:25.958160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:25.960408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:25.987418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1118/2216] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:27.232708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a7c6-6da5-4cff-a8f9-45999d9088f1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:27.233255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a7c6-6da5-4cff-a8f9-45999d9088f1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:27.233357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613a7c6-6da5-4cff-a8f9-45999d9088f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:27.266293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1613a7c6-6da5-4cff-a8f9-45999d9088f1 None None] MFA Rules not processed for user `2ab9930d6ce54cacbb3d4fd6313294d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:27.267886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1613a7c6-6da5-4cff-a8f9-45999d9088f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:27.323685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:27.367639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1099/2217] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:27.372579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:27.374032 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:27.464347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbdad8-8017-409a-82c7-2fbdc9ec428a service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:27.472097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:27.472882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:27.473173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:27.473451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:27.474123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:27.476105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:27.519689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1119/2218] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:27 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:27.521174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a7c6-6da5-4cff-a8f9-45999d9088f1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:27.536330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613a7c6-6da5-4cff-a8f9-45999d9088f1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:27.625149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:27.625779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:27.626059 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:27.626334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:27.626997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:27.629449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:27.661817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1100/2219] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:27 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:27.666154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:27.679390 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9baa1e-ba8b-4432-a524-bdf2c83ee0e6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:27.684795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:27.685424 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:27.685700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:27.685967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:27.686610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:27.688235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:27.701149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1120/2220] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:27.705598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:27.716785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905b5e9-2caf-476d-9fbb-d0472bbd5a96 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:27.721667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:27.722329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:27.722603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:27.722872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] PATH_INFO: `/v3/projects/1e4aa136d4384ac7a9bcb795533eec73/users/6f3e442d656f4e6eb1eaf367e4c70238/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:27.740588 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4aa136d4384ac7a9bcb795533eec73, user_id=6f3e442d656f4e6eb1eaf367e4c70238, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:27.743453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:27.761936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1101/2221] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:27 2022] PUT /identity/v3/projects/1e4aa136d4384ac7a9bcb795533eec73/users/6f3e442d656f4e6eb1eaf367e4c70238/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:27.765255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:27.777107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963fc5a-9452-4a6d-b886-c5225ab14dc4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:27.879041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:27.888103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:27.888601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:27.889799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:27.890156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:27.891939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:27.911478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1121/2222] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:27.915851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:27.928278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32cd74-ebc3-44f5-98a9-dac93d99978c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:27.937000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:27.937842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:27.938238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:27.938638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] PATH_INFO: `/v3/projects/1e4aa136d4384ac7a9bcb795533eec73/users/6f3e442d656f4e6eb1eaf367e4c70238/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:27.941289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4aa136d4384ac7a9bcb795533eec73, user_id=6f3e442d656f4e6eb1eaf367e4c70238, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:27.944469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:27.956520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1102/2223] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:27 2022] PUT /identity/v3/projects/1e4aa136d4384ac7a9bcb795533eec73/users/6f3e442d656f4e6eb1eaf367e4c70238/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:28.213628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:28.245928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:28.296617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e9f5c-87c3-46a5-8850-9615b1d24081 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:28.417145 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3ccb2-4eee-494a-96b8-fd04df162e07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:28.418696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ccb2-4eee-494a-96b8-fd04df162e07 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:28.419152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ccb2-4eee-494a-96b8-fd04df162e07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:28.419839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ccb2-4eee-494a-96b8-fd04df162e07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:28.552388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3ccb2-4eee-494a-96b8-fd04df162e07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:28.553901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3ccb2-4eee-494a-96b8-fd04df162e07 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:28.569110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1122/2224] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:32.030896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:32.174514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e464ec7-c23b-4343-bcb4-09f6ea99c9de admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:32.185714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:32.186060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:32.186157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:32.186612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] PATH_INFO: `/v3/users/b03e7a8ab6c34868af811ade7426e09c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:32.192885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03e7a8ab6c34868af811ade7426e09c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:32.209442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:32.278424 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:32.312894 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03e7a8ab6c34868af811ade7426e09c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:32.363399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03e7a8ab6c34868af811ade7426e09c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:32.364443 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03e7a8ab6c34868af811ade7426e09c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:32.383801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03e7a8ab6c34868af811ade7426e09c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:32.433120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03e7a8ab6c34868af811ade7426e09c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:32.443618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1103/2225] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:32 2022] DELETE /identity/v3/users/b03e7a8ab6c34868af811ade7426e09c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:32.448985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:32.449510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:32.450190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:32.475334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:32.477766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:32.562561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1123/2226] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:32.574352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:32.576133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:32.650085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56309b8e-a04e-4d48-99fb-72dcba4a5fd2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:32.655898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:32.657702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:32.657702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:32.657702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:32.669941 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:32.672328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:32.692892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1104/2227] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:33.231599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:33.233201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:33.249999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7ff92-1cf9-4f9a-89d5-e7c462aeb43f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:33.255982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:33.256943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.257497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.257779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] PATH_INFO: `/v3/users/ae2edac9b70e4f83ab6734479d359f7f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.259024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae2edac9b70e4f83ab6734479d359f7f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:33.261052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:33.280939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:33.282397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:33.291196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:33.300918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24641fa0-e9b2-427d-81f8-c81aedf9aba4 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:33.307342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:33.308061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.308344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.308760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] PATH_INFO: `/v3/projects/d8dea26ee3b04f11a491c5d2fc843350` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.310516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8dea26ee3b04f11a491c5d2fc843350)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:33.312564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:33.366185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae2edac9b70e4f83ab6734479d359f7f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.380322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae2edac9b70e4f83ab6734479d359f7f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.381151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae2edac9b70e4f83ab6734479d359f7f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.388535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8dea26ee3b04f11a491c5d2fc843350', 'request_id': 'req-d3ede364-db2d-4162-9668-65d6116e01fe'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.391064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae2edac9b70e4f83ab6734479d359f7f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.406208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e366aa4f-64b6-4acc-a7ea-6117d2c2468b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae2edac9b70e4f83ab6734479d359f7f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.413707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8dea26ee3b04f11a491c5d2fc843350', 'request_id': 'req-d3ede364-db2d-4162-9668-65d6116e01fe'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.414227 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1124/2228] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:33 2022] DELETE /identity/v3/users/ae2edac9b70e4f83ab6734479d359f7f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:33.418400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.420599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.433320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.448222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8dea26ee3b04f11a491c5d2fc843350', 'request_id': 'req-d3ede364-db2d-4162-9668-65d6116e01fe'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.469622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] The token cache is being invalidate because project d8dea26ee3b04f11a491c5d2fc843350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:33.470086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.470864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d3ede364-db2d-4162-9668-65d6116e01fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:33.476838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1105/2229] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:33 2022] DELETE /identity/v3/projects/d8dea26ee3b04f11a491c5d2fc843350 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:33.478533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845c7a-62a8-4cae-a1c4-a50f84391ef5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.484071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845c7a-62a8-4cae-a1c4-a50f84391ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.484535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845c7a-62a8-4cae-a1c4-a50f84391ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.491638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:33.495496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:33.517964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-31845c7a-62a8-4cae-a1c4-a50f84391ef5 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:33.519387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-31845c7a-62a8-4cae-a1c4-a50f84391ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:33.597357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:33.638452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1125/2230] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:33.662338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:33.664052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:33.679097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1106/2231] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:33.681316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0358b1c-cc02-4f48-9b02-3a3eddc78d00 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.681606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0358b1c-cc02-4f48-9b02-3a3eddc78d00 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.681889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0358b1c-cc02-4f48-9b02-3a3eddc78d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.738904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d0358b1c-cc02-4f48-9b02-3a3eddc78d00 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:33.740254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d0358b1c-cc02-4f48-9b02-3a3eddc78d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:33.764824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1107/2232] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:33.771282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0358b1c-cc02-4f48-9b02-3a3eddc78d00 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:33.789987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0358b1c-cc02-4f48-9b02-3a3eddc78d00 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:33.806802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:33.809621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.810344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.812352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.814925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:33.815418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:33.819694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:33.832634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18b4d8-d6a6-4495-a39b-b188be777db5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:33.849124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc80fe-1843-46d0-8ad7-85700ddce06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:33.851782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc80fe-1843-46d0-8ad7-85700ddce06e service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.853883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc80fe-1843-46d0-8ad7-85700ddce06e service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.855839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc80fe-1843-46d0-8ad7-85700ddce06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.861562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1108/2233] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:33.866904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:33.883715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c384c4-465b-4787-995b-f380f2e617fe admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:33.921399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc80fe-1843-46d0-8ad7-85700ddce06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:33.923855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:33.925771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.927734 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.928850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc80fe-1843-46d0-8ad7-85700ddce06e service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:33.929340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:33.931292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:33.950855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:33.981293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1126/2234] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:33.983851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:33.984218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:33.984544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.028568 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1109/2235] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:33 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:34.038353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.081512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] MFA Rules not processed for user `6f3e442d656f4e6eb1eaf367e4c70238`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:34.086870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:34.155376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:34.232642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c21f-a53a-4b2f-8b2e-6e5c14275e84 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:34.241032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:34.241658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.242169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.242330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.243049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:34.245024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:34.281031 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1127/2236] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:34.305560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.306587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1110/2237] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:34 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:34.311146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.311337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:34.334630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bde1c-8583-4562-b247-7ce0cb8793f3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:34.346875 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:34.347620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.348251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.348833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.349263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:34.351282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:34.379475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1111/2238] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:34.383407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.406308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecd3f8-435a-4465-b626-58dac2ab643b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:34.415110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:34.416201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.416694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.417151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.418259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:34.423665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:34.465638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1112/2239] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:34.469694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.486815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639af83-2b39-4f92-9a78-a6096332eb03 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:34.495223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d936ccf-6b91-4f61-928c-017f068dc537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:34.496314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d936ccf-6b91-4f61-928c-017f068dc537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.496873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d936ccf-6b91-4f61-928c-017f068dc537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.497220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d936ccf-6b91-4f61-928c-017f068dc537 admin admin] PATH_INFO: `/v3/projects/21e224cf1aec4e68b76880e701ed6f26/users/301abe5d4fb9434098eb8e7309b3b64b/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.498527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c1884-85db-4277-a61f-d40f4a5ff4c8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:34.512577 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:34.513572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.513855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.514130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.516734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:34.517639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d936ccf-6b91-4f61-928c-017f068dc537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e224cf1aec4e68b76880e701ed6f26, user_id=301abe5d4fb9434098eb8e7309b3b64b, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:34.520789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d936ccf-6b91-4f61-928c-017f068dc537 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:34.548281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1113/2240] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:34 2022] PUT /identity/v3/projects/21e224cf1aec4e68b76880e701ed6f26/users/301abe5d4fb9434098eb8e7309b3b64b/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:34.559541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b1bfb-6f38-46db-bdac-8a2b757442d6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.559938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b1bfb-6f38-46db-bdac-8a2b757442d6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.560375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b1bfb-6f38-46db-bdac-8a2b757442d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.615810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:34.617093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-583b1bfb-6f38-46db-bdac-8a2b757442d6 None None] MFA Rules not processed for user `301abe5d4fb9434098eb8e7309b3b64b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:34.617461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:34.619003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-583b1bfb-6f38-46db-bdac-8a2b757442d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:34.635887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1128/2241] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:34.637222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.639440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:34.645454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:34.656127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d85e-1ca6-46c8-9c62-1a2980455f26 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:34.738407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1114/2242] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:34.743955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b1bfb-6f38-46db-bdac-8a2b757442d6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.783037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:34.783769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.784039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.784361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] PATH_INFO: `/v3/projects/51bcb82c807b4a22a80a2bb9df888c5a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.785801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51bcb82c807b4a22a80a2bb9df888c5a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:34.788663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:34.797614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:34.864855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b1bfb-6f38-46db-bdac-8a2b757442d6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:34.865721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51bcb82c807b4a22a80a2bb9df888c5a', 'request_id': 'req-d9a7f316-b759-4b8b-8d67-fb5b711ee231'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:34.887505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51bcb82c807b4a22a80a2bb9df888c5a', 'request_id': 'req-d9a7f316-b759-4b8b-8d67-fb5b711ee231'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:34.895933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51bcb82c807b4a22a80a2bb9df888c5a', 'request_id': 'req-d9a7f316-b759-4b8b-8d67-fb5b711ee231'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:34.909914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] The token cache is being invalidate because project 51bcb82c807b4a22a80a2bb9df888c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:34.910497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:34.913362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:34.919198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1129/2243] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:34 2022] DELETE /identity/v3/projects/51bcb82c807b4a22a80a2bb9df888c5a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:34.923447 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:34.925020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:34.991086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:34.991086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:34.991086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:34.991086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:34.991499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:35.005505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:35.111741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1115/2244] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:34 2022] POST /identity/v3/projects => generated 363 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:35.115179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:35.118882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:35.143434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7f316-b759-4b8b-8d67-fb5b711ee231 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:35.281846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:35.297254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:35.297841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:35.298122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:35.298397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] PATH_INFO: `/v3/users/465fc5f283bb43ab9581405e2ed22156` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:35.302387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] RBAC: Authorizing `identity:delete_user(user_id=465fc5f283bb43ab9581405e2ed22156)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:35.302994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dc25d-955b-4994-8f5b-2785fce9eb5c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:35.304396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:35.323476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:35.326942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:35.327291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:35.327900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:35.333627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:35.338991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:35.385353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:35.391973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '465fc5f283bb43ab9581405e2ed22156'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:35.396489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1116/2245] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:35 2022] POST /identity/v3/users => generated 288 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:35.402027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:35.406621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '465fc5f283bb43ab9581405e2ed22156'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:35.407551 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '465fc5f283bb43ab9581405e2ed22156'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:35.432374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeb190-b138-4197-992e-576d9d6ef171 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:35.433544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '465fc5f283bb43ab9581405e2ed22156'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:35.448050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:35.448715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:35.448992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:35.449261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:35.451204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:35.452901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:35.464354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '465fc5f283bb43ab9581405e2ed22156'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:35.474902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1117/2246] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:35.479310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1130/2247] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:34 2022] DELETE /identity/v3/users/465fc5f283bb43ab9581405e2ed22156 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:35.480821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:35.482620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:35.501545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef69b-5e8d-4e88-b8b9-0587a04747de admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:35.633042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:35.634042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:35.634212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:35.634481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] PATH_INFO: `/v3/projects/60647f765b2f4a32a314a0351503f750/users/8008bff5fb0043c78303e089638186be/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:35.654446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60647f765b2f4a32a314a0351503f750, user_id=8008bff5fb0043c78303e089638186be, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:35.659801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:35.697544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1118/2248] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:35 2022] PUT /identity/v3/projects/60647f765b2f4a32a314a0351503f750/users/8008bff5fb0043c78303e089638186be/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:35.701148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:35.727732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01537-4f92-4283-a524-1dbde2aacabd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:35.831574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9e083-7b6d-41ab-b4a0-bc8cfb6f3396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:35.832485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9e083-7b6d-41ab-b4a0-bc8cfb6f3396 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:35.832834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9e083-7b6d-41ab-b4a0-bc8cfb6f3396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:35.833180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9e083-7b6d-41ab-b4a0-bc8cfb6f3396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:35.833973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9e083-7b6d-41ab-b4a0-bc8cfb6f3396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:35.836462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9e083-7b6d-41ab-b4a0-bc8cfb6f3396 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:35.849137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1131/2249] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:35.853536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:35.872315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4170c-3280-44ba-a0d4-d89491de237b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:35.889849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:35.893690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:35.893690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:35.893690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] PATH_INFO: `/v3/projects/60647f765b2f4a32a314a0351503f750/users/8008bff5fb0043c78303e089638186be/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:35.897770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=60647f765b2f4a32a314a0351503f750, user_id=8008bff5fb0043c78303e089638186be, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:35.905212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:35.944222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:35.944627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1119/2250] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:35 2022] PUT /identity/v3/projects/60647f765b2f4a32a314a0351503f750/users/8008bff5fb0043c78303e089638186be/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:35.944928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:35.945370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:36.029165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] MFA Rules not processed for user `8008bff5fb0043c78303e089638186be`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:36.030505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:36.108042 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:36.110667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:36.254024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1132/2251] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:36.263015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d45f0-fe6f-4591-b40c-14afa8dc2dbe admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:36.269425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:36.269535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:36.269661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:36.269766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:36.269868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:36.270168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] PATH_INFO: `/v3/projects/3d8f2b51eda34f3f8aecc84e9c770300` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:36.271528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8f2b51eda34f3f8aecc84e9c770300)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:36.274066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:36.359260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:36.378747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:36.384711 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8f2b51eda34f3f8aecc84e9c770300', 'request_id': 'req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.387109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8d0b4-dd00-4162-b4bd-46d6cf1aac98 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:36.400112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8f2b51eda34f3f8aecc84e9c770300', 'request_id': 'req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.426243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8f2b51eda34f3f8aecc84e9c770300', 'request_id': 'req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.444331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] The token cache is being invalidate because project 3d8f2b51eda34f3f8aecc84e9c770300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:36.444989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.446403 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.450646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1120/2252] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:36 2022] DELETE /identity/v3/projects/3d8f2b51eda34f3f8aecc84e9c770300 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:36.455523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:36.458802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:36.561468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27aabb-aa61-4cc8-a0a7-712970be755e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:36.562350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27aabb-aa61-4cc8-a0a7-712970be755e service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:36.562813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27aabb-aa61-4cc8-a0a7-712970be755e service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:36.563331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27aabb-aa61-4cc8-a0a7-712970be755e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:36.568033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3e27aabb-aa61-4cc8-a0a7-712970be755e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:36.615158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4ab03-5c09-44fa-9e66-8ff0f0bca25d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:36.640509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:36.640509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:36.640509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:36.640509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] PATH_INFO: `/v3/users/60eac013f2494c3fb6f4c3d21fcdfd8d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:36.641141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60eac013f2494c3fb6f4c3d21fcdfd8d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:36.641371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:36.715639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27aabb-aa61-4cc8-a0a7-712970be755e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:36.721662 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27aabb-aa61-4cc8-a0a7-712970be755e service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:36.744715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60eac013f2494c3fb6f4c3d21fcdfd8d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.749555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1133/2253] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:36.760567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60eac013f2494c3fb6f4c3d21fcdfd8d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.761252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60eac013f2494c3fb6f4c3d21fcdfd8d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.781756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60eac013f2494c3fb6f4c3d21fcdfd8d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.807471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60eac013f2494c3fb6f4c3d21fcdfd8d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:36.820659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1121/2254] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:36 2022] DELETE /identity/v3/users/60eac013f2494c3fb6f4c3d21fcdfd8d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:36.830837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:36.831135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:36.831413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:36.872923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] MFA Rules not processed for user `2ab9930d6ce54cacbb3d4fd6313294d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:36.874198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:36.918307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:36.919821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:36.985315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1134/2255] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:37.017048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e891602-c766-40f7-8b76-9d41350c67a3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:37.022939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea537c-4e13-4f5e-ae55-3308d29f6ab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:37.023626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea537c-4e13-4f5e-ae55-3308d29f6ab7 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:37.023953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea537c-4e13-4f5e-ae55-3308d29f6ab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:37.024243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea537c-4e13-4f5e-ae55-3308d29f6ab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:37.029228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-28ea537c-4e13-4f5e-ae55-3308d29f6ab7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:37.037203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:37.104035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea537c-4e13-4f5e-ae55-3308d29f6ab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:37.110220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea537c-4e13-4f5e-ae55-3308d29f6ab7 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:37.124581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1122/2256] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:03:37.202810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:37.206216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:37.287571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:37.305746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8869e-aba5-4fb1-bbf2-c11484931a8d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:37.311704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:37.312293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:37.312890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:37.313135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] PATH_INFO: `/v3/projects/6778af1293444c37b5b0c9c0813f0950` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:37.314753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6778af1293444c37b5b0c9c0813f0950)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:37.317570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:37.411005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6778af1293444c37b5b0c9c0813f0950', 'request_id': 'req-d465e55c-0f3e-413f-9640-907ddb2bf376'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:37.433001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6778af1293444c37b5b0c9c0813f0950', 'request_id': 'req-d465e55c-0f3e-413f-9640-907ddb2bf376'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:37.441428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6778af1293444c37b5b0c9c0813f0950', 'request_id': 'req-d465e55c-0f3e-413f-9640-907ddb2bf376'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:37.457925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] The token cache is being invalidate because project 6778af1293444c37b5b0c9c0813f0950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:37.458285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:37.459038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d465e55c-0f3e-413f-9640-907ddb2bf376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:37.460982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1135/2257] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:37 2022] DELETE /identity/v3/projects/6778af1293444c37b5b0c9c0813f0950 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:38.289452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1df9-d885-4159-9196-364b4916fd1d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:38.290379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1df9-d885-4159-9196-364b4916fd1d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:38.290857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1df9-d885-4159-9196-364b4916fd1d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:38.292673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1123/2258] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:03:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:03:38.308008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:38.308675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:38.308675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:38.335244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:38.336781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:38.444451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1136/2259] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:03:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:38.493740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1df9-d885-4159-9196-364b4916fd1d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:38.496144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3e7e1df9-d885-4159-9196-364b4916fd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:38.528292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1df9-d885-4159-9196-364b4916fd1d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:38.539477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:38.540310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:38.541297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:38.542567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:38.560523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:38.562970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:38.620929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1124/2260] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:39.126527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:39.128489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:39.221125 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67731c70-a59a-4db5-9b0e-ddf1919c42f9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:39.228939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:39.230307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:39.231357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:39.231994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] PATH_INFO: `/v3/users/373b7e558c464082ab451c5dd6b361cc` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:39.233582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=373b7e558c464082ab451c5dd6b361cc)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:39.236336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:39.272975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:39.304003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '373b7e558c464082ab451c5dd6b361cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.316742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '373b7e558c464082ab451c5dd6b361cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.318310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '373b7e558c464082ab451c5dd6b361cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.328615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '373b7e558c464082ab451c5dd6b361cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.342345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '373b7e558c464082ab451c5dd6b361cc'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.349322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1137/2261] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:39 2022] DELETE /identity/v3/users/373b7e558c464082ab451c5dd6b361cc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:39.540150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:39.551604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bc530-3de3-45a6-a70b-1960312749af service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:39.615807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:39.616449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:39.616731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:39.616999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] PATH_INFO: `/v3/projects/f350884ae98c454aac0c1e6750d06bcb` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:39.618230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f350884ae98c454aac0c1e6750d06bcb)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:39.620162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:39.688220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f350884ae98c454aac0c1e6750d06bcb', 'request_id': 'req-7ffd165c-7818-4f2a-8559-753c0728387a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.697367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f350884ae98c454aac0c1e6750d06bcb', 'request_id': 'req-7ffd165c-7818-4f2a-8559-753c0728387a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.703991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f350884ae98c454aac0c1e6750d06bcb', 'request_id': 'req-7ffd165c-7818-4f2a-8559-753c0728387a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.714772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] The token cache is being invalidate because project f350884ae98c454aac0c1e6750d06bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:39.715064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.715824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.717367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1125/2262] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:39 2022] DELETE /identity/v3/projects/f350884ae98c454aac0c1e6750d06bcb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:39.721568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:39.723242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:39.739086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eba8e-0f44-4f66-9662-0ced3fd5aecf admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:39.811917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:39.812603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:39.812886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:39.813158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] PATH_INFO: `/v3/users/93a04ec0d9894df582743ea36bd8fede` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:39.814312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a04ec0d9894df582743ea36bd8fede)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:39.816304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:39.887714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a04ec0d9894df582743ea36bd8fede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.898392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a04ec0d9894df582743ea36bd8fede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.899284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a04ec0d9894df582743ea36bd8fede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.912689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a04ec0d9894df582743ea36bd8fede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.924409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bae7a6fe-9406-413c-a9cf-23da85edf67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a04ec0d9894df582743ea36bd8fede'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:39.930878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1138/2263] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:39 2022] DELETE /identity/v3/users/93a04ec0d9894df582743ea36bd8fede => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:40.184411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:40.186510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:40.215949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd165c-7818-4f2a-8559-753c0728387a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:40.245539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:40.314094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:40.317069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:40.317374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:40.317642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] PATH_INFO: `/v3/projects/398d9e1e85a9420f850746df7f796910` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:40.322186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=398d9e1e85a9420f850746df7f796910)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:40.324342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:40.407666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398d9e1e85a9420f850746df7f796910', 'request_id': 'req-bafa3f06-5ff4-49b7-8186-093bbe951448'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:40.421795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398d9e1e85a9420f850746df7f796910', 'request_id': 'req-bafa3f06-5ff4-49b7-8186-093bbe951448'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:40.440192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398d9e1e85a9420f850746df7f796910', 'request_id': 'req-bafa3f06-5ff4-49b7-8186-093bbe951448'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:40.458249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] The token cache is being invalidate because project 398d9e1e85a9420f850746df7f796910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:40.461845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:40.463134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bafa3f06-5ff4-49b7-8186-093bbe951448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:40.465478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1126/2264] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:40 2022] DELETE /identity/v3/projects/398d9e1e85a9420f850746df7f796910 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:40.470420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dcd3b-d4ac-4fc7-8cbf-c98098ae155a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:40.470921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dcd3b-d4ac-4fc7-8cbf-c98098ae155a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:40.471010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dcd3b-d4ac-4fc7-8cbf-c98098ae155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:40.498625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-201dcd3b-d4ac-4fc7-8cbf-c98098ae155a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:40.502748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-201dcd3b-d4ac-4fc7-8cbf-c98098ae155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:40.521869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:40.606794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1139/2265] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:40.615508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fef748-7038-4f30-82c7-921f459abc9b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:40.616084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fef748-7038-4f30-82c7-921f459abc9b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:40.616395 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fef748-7038-4f30-82c7-921f459abc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:40.647881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f5fef748-7038-4f30-82c7-921f459abc9b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:40.649946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f5fef748-7038-4f30-82c7-921f459abc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:40.673510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1127/2266] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:40.679540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dcd3b-d4ac-4fc7-8cbf-c98098ae155a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:40.702061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dcd3b-d4ac-4fc7-8cbf-c98098ae155a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:40.739141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:40.740015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:40.740478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:40.742252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:40.743286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:40.743796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:40.745860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:40.771648 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1140/2267] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:40.776990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fef748-7038-4f30-82c7-921f459abc9b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:40.790255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fef748-7038-4f30-82c7-921f459abc9b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:40.795421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:40.796037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:40.796316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:40.796620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:40.797274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:40.799349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:40.832895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1128/2268] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:40 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:40.837338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:40.847912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e717d7-3366-4056-9d96-94d1486de3e5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:40.906626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:40.907296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:40.907500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:40.907794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:40.908452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:40.910418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:40.940943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1141/2269] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:40 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:40.945082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:40.957482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161e754-56b4-4b0e-be46-da3ea31619e1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:40.966369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:40.967188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:40.967464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:40.967746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:40.968397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:40.970177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:40.982123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1129/2270] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:40.985875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:40.996637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1015849-f8b7-4481-8684-e6cb273f1429 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:41.003602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83623031-3819-432f-a6ba-ac74865ade64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:41.004204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83623031-3819-432f-a6ba-ac74865ade64 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:41.004568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83623031-3819-432f-a6ba-ac74865ade64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:41.004845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83623031-3819-432f-a6ba-ac74865ade64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:41.005594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83623031-3819-432f-a6ba-ac74865ade64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:41.007282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83623031-3819-432f-a6ba-ac74865ade64 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:41.020185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1142/2271] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:41.024673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:41.038425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7078f66-6de1-4d6e-9017-59baaaa3f0ce admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:41.048383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:41.049334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:41.049609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:41.050533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] PATH_INFO: `/v3/projects/9e6a142d8e574888a3af29f95b99a14e/users/a76682f0853842a8b27906ba09edb527/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:41.073654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6a142d8e574888a3af29f95b99a14e, user_id=a76682f0853842a8b27906ba09edb527, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:41.076731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:41.093355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1130/2272] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:41 2022] PUT /identity/v3/projects/9e6a142d8e574888a3af29f95b99a14e/users/a76682f0853842a8b27906ba09edb527/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:41.107764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:41.107764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:41.107764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:41.137166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] MFA Rules not processed for user `a76682f0853842a8b27906ba09edb527`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:41.138395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:41.210575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1143/2273] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:41.222937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:41.225083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:41.284240 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df631c90-ab41-47f6-80e3-27656e3422fd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:41.287964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786bc6c-b612-42db-8809-4fb010ca3894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:41.288654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786bc6c-b612-42db-8809-4fb010ca3894 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:41.288923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786bc6c-b612-42db-8809-4fb010ca3894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:41.289189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786bc6c-b612-42db-8809-4fb010ca3894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:41.294272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:41.299182 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786bc6c-b612-42db-8809-4fb010ca3894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:41.300677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786bc6c-b612-42db-8809-4fb010ca3894 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:41.310455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1131/2274] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:45.324037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:45.326132 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:45.341986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:45.401397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2eecc-a78f-4811-9750-befd7314fb5f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:45.411235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:45.411235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:45.411235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:45.411235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] PATH_INFO: `/v3/users/8a7863895c8848258368f13ffef7812d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:45.411517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7863895c8848258368f13ffef7812d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:45.411517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:45.471695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7863895c8848258368f13ffef7812d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:45.484398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7863895c8848258368f13ffef7812d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:45.485066 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7863895c8848258368f13ffef7812d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:45.493370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7863895c8848258368f13ffef7812d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:45.509555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7863895c8848258368f13ffef7812d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:45.519671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1144/2275] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:45 2022] DELETE /identity/v3/users/8a7863895c8848258368f13ffef7812d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:45.524278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:45.524822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:45.525908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:45.553815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:45.555389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:45.654970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1132/2276] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:45.666407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:45.667638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:45.768607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc38977-e99a-450b-9812-20e3af5e2eb5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:45.773002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:45.773883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:45.774370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:45.775036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:45.789191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:45.790634 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:45.803009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1145/2277] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:46.045280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:46.046812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:46.059023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecac75c-424f-446b-81ab-45305edc30ae None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:46.063649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:46.064301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:46.064615 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:46.064917 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] PATH_INFO: `/v3/users/a76682f0853842a8b27906ba09edb527` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:46.066010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76682f0853842a8b27906ba09edb527)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:46.068023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:46.108845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:46.110637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:46.122491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76682f0853842a8b27906ba09edb527'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.125812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4381709-fd48-4287-a71f-8c37a993adea service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:46.132111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:46.132919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:46.133274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76682f0853842a8b27906ba09edb527'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.133391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:46.133917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] PATH_INFO: `/v3/projects/17055643a8d444b48a343ea95ee61bbd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:46.134079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76682f0853842a8b27906ba09edb527'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.135609 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17055643a8d444b48a343ea95ee61bbd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:46.138298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:46.142905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76682f0853842a8b27906ba09edb527'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.161131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e644a249-f3af-44d3-808e-1733a61160ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76682f0853842a8b27906ba09edb527'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.167540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1133/2278] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:46 2022] DELETE /identity/v3/users/a76682f0853842a8b27906ba09edb527 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:46.173050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:46.173334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:46.173608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:46.177587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:46.192958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17055643a8d444b48a343ea95ee61bbd', 'request_id': 'req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.193851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:46.195437 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:46.208007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17055643a8d444b48a343ea95ee61bbd', 'request_id': 'req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.214631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17055643a8d444b48a343ea95ee61bbd', 'request_id': 'req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.226169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] The token cache is being invalidate because project 17055643a8d444b48a343ea95ee61bbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:46.226459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.227112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.228914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1146/2279] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:46 2022] DELETE /identity/v3/projects/17055643a8d444b48a343ea95ee61bbd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:46.271278 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1134/2280] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:46.282061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:46.283664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:46.355182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1773e6-9d4f-4ffd-ab86-7927ef8de4d3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:46.359002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b1a26-6583-423b-9390-72f6359103b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:46.359604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1a26-6583-423b-9390-72f6359103b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:46.359876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1a26-6583-423b-9390-72f6359103b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:46.360185 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1a26-6583-423b-9390-72f6359103b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:46.371065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b1a26-6583-423b-9390-72f6359103b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:46.372792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b1a26-6583-423b-9390-72f6359103b6 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:46.384222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1147/2281] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:46.677261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:46.679850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:46.695255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e8906-bdd6-4b92-86cc-7803d0a9e690 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:46.700692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:46.701348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:46.701620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:46.701888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] PATH_INFO: `/v3/projects/9e6a142d8e574888a3af29f95b99a14e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:46.703062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e6a142d8e574888a3af29f95b99a14e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:46.711992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:46.714047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:46.774074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e6a142d8e574888a3af29f95b99a14e', 'request_id': 'req-12f48a77-cf7b-4c48-b711-eb68ab3def19'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.785423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e6a142d8e574888a3af29f95b99a14e', 'request_id': 'req-12f48a77-cf7b-4c48-b711-eb68ab3def19'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.792799 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e6a142d8e574888a3af29f95b99a14e', 'request_id': 'req-12f48a77-cf7b-4c48-b711-eb68ab3def19'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.803863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] The token cache is being invalidate because project 9e6a142d8e574888a3af29f95b99a14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:46.804138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.804853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-12f48a77-cf7b-4c48-b711-eb68ab3def19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:46.806443 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1135/2282] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:46 2022] DELETE /identity/v3/projects/9e6a142d8e574888a3af29f95b99a14e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:46.811428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141edf7a-f4f4-49d3-bca2-64ed009969ee None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:46.811849 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141edf7a-f4f4-49d3-bca2-64ed009969ee None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:46.811995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141edf7a-f4f4-49d3-bca2-64ed009969ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:46.844451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-141edf7a-f4f4-49d3-bca2-64ed009969ee None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:46.846105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-141edf7a-f4f4-49d3-bca2-64ed009969ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:46.934763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1148/2283] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:46.944741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda321d9-5c02-4850-8b3d-baef664cfe4a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:46.960452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda321d9-5c02-4850-8b3d-baef664cfe4a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:46.960452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda321d9-5c02-4850-8b3d-baef664cfe4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:46.974354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cda321d9-5c02-4850-8b3d-baef664cfe4a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:46.975726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cda321d9-5c02-4850-8b3d-baef664cfe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:46.988958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1136/2284] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:46.991700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141edf7a-f4f4-49d3-bca2-64ed009969ee None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:47.003331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141edf7a-f4f4-49d3-bca2-64ed009969ee None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:47.008566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:47.009308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.009759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.010434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.010991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:47.011394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:47.013143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:47.032558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1149/2285] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:03:47.037676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda321d9-5c02-4850-8b3d-baef664cfe4a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:47.050058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda321d9-5c02-4850-8b3d-baef664cfe4a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:47.056118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:47.058555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.059219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.060228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.060989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:47.063531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:47.115311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1137/2286] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:47 2022] POST /identity/v3/projects => generated 397 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:47.121700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:47.134494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451d4f8-7a94-40ed-bc40-69585a14e2d9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:47.194420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:47.210857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:47.211658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.211955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.212232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.212991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:47.215088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:47.239743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1150/2287] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:47 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:47.247408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:47.258731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83ae5f-76d9-4e1c-9147-206ba7ffeb11 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:47.264138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:47.264910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.265260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.265589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.266328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:47.268080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:47.279747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1138/2288] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:47.284242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:47.295222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1d32a-0e6a-4397-b369-048500767d22 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:47.301284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1acbe-c75e-4c13-a51b-e297e1178450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:47.302041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1acbe-c75e-4c13-a51b-e297e1178450 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.302323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1acbe-c75e-4c13-a51b-e297e1178450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.302595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1acbe-c75e-4c13-a51b-e297e1178450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.303310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1acbe-c75e-4c13-a51b-e297e1178450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:47.305107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1acbe-c75e-4c13-a51b-e297e1178450 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:47.317520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1151/2289] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:47.322808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:47.335763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e080-7026-465a-a5b9-16c6291282c7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:47.341141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:47.341971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.342338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.342688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] PATH_INFO: `/v3/projects/77667d7dcad04b1aaaae178536cdf127/users/dd9d4069961a47c78351786a7aef4ad4/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.359460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77667d7dcad04b1aaaae178536cdf127, user_id=dd9d4069961a47c78351786a7aef4ad4, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:47.362334 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:47.387736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1139/2290] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:47 2022] PUT /identity/v3/projects/77667d7dcad04b1aaaae178536cdf127/users/dd9d4069961a47c78351786a7aef4ad4/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:47.390129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.390692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.390789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.430908 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] MFA Rules not processed for user `dd9d4069961a47c78351786a7aef4ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:47.432385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:47.539000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1152/2291] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:47.551577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:47.554987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:47.646689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a782b-753b-44ec-933d-3fe9265a4be0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:47.651000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4c27f-fee6-4061-a345-3baa43b17af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:47.651679 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4c27f-fee6-4061-a345-3baa43b17af4 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:47.651999 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4c27f-fee6-4061-a345-3baa43b17af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:47.652276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4c27f-fee6-4061-a345-3baa43b17af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:47.663201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4c27f-fee6-4061-a345-3baa43b17af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:47.664937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4c27f-fee6-4061-a345-3baa43b17af4 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:47.675629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1140/2292] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:48.367298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:48.368707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:48.437515 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199a9e6-00d6-4a34-92c8-f0e578ff6423 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:48.442411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:48.443067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:48.443343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:48.443614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] PATH_INFO: `/v3/users/301abe5d4fb9434098eb8e7309b3b64b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:48.444784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=301abe5d4fb9434098eb8e7309b3b64b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:48.446745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:48.497517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301abe5d4fb9434098eb8e7309b3b64b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:48.507541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301abe5d4fb9434098eb8e7309b3b64b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:48.508393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301abe5d4fb9434098eb8e7309b3b64b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:48.514983 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:48.519595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301abe5d4fb9434098eb8e7309b3b64b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:48.534162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301abe5d4fb9434098eb8e7309b3b64b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:48.541263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1153/2293] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:48 2022] DELETE /identity/v3/users/301abe5d4fb9434098eb8e7309b3b64b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:48.546152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f3257-3ce1-4951-b919-4b014586120d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:48.546695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f3257-3ce1-4951-b919-4b014586120d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:48.546695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f3257-3ce1-4951-b919-4b014586120d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:48.567136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-295f3257-3ce1-4951-b919-4b014586120d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:48.568330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-295f3257-3ce1-4951-b919-4b014586120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:48.595977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:48.634370 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1141/2294] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:48.644505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:48.646141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:48.711752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b9495-ea65-4494-be88-fef6bf06c1ec admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:48.715433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:48.716039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:48.716313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:48.716610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:48.728222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:48.730085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:48.741202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1154/2295] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:49.031627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f3257-3ce1-4951-b919-4b014586120d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:49.033157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-295f3257-3ce1-4951-b919-4b014586120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:49.046084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f3257-3ce1-4951-b919-4b014586120d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:49.052520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:49.053453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:49.053801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:49.054140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] PATH_INFO: `/v3/projects/21e224cf1aec4e68b76880e701ed6f26` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:49.055477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e224cf1aec4e68b76880e701ed6f26)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:49.057794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:49.110426 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e224cf1aec4e68b76880e701ed6f26', 'request_id': 'req-d9171f51-542a-4b57-b70b-415c30b7e885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.119884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e224cf1aec4e68b76880e701ed6f26', 'request_id': 'req-d9171f51-542a-4b57-b70b-415c30b7e885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.127472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e224cf1aec4e68b76880e701ed6f26', 'request_id': 'req-d9171f51-542a-4b57-b70b-415c30b7e885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.138853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] The token cache is being invalidate because project 21e224cf1aec4e68b76880e701ed6f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:49.139324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.140274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.141975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1142/2296] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:49 2022] DELETE /identity/v3/projects/21e224cf1aec4e68b76880e701ed6f26 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:49.146836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:49.158074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f5e2a-8176-4714-85bf-cab462b1de6b service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:49.225398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:49.226034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:49.226716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:49.226716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] PATH_INFO: `/v3/users/8008bff5fb0043c78303e089638186be` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:49.227691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8008bff5fb0043c78303e089638186be)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:49.229807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:49.294384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8008bff5fb0043c78303e089638186be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.302687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8008bff5fb0043c78303e089638186be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.304766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8008bff5fb0043c78303e089638186be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.314477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8008bff5fb0043c78303e089638186be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.328839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-36ca47c9-44b6-429b-a62e-918de6e51d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8008bff5fb0043c78303e089638186be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.331460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:49.336048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1155/2297] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:49 2022] DELETE /identity/v3/users/8008bff5fb0043c78303e089638186be => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:49.738106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:49.739690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:49.756363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9171f51-542a-4b57-b70b-415c30b7e885 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:49.815416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:49.816062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:49.816340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:49.816644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] PATH_INFO: `/v3/projects/60647f765b2f4a32a314a0351503f750` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:49.817878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60647f765b2f4a32a314a0351503f750)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:49.819869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:49.881798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60647f765b2f4a32a314a0351503f750', 'request_id': 'req-10deb520-17e0-4041-aeac-079b65774245'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.886569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:49.892669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60647f765b2f4a32a314a0351503f750', 'request_id': 'req-10deb520-17e0-4041-aeac-079b65774245'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.900603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60647f765b2f4a32a314a0351503f750', 'request_id': 'req-10deb520-17e0-4041-aeac-079b65774245'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.917051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] The token cache is being invalidate because project 60647f765b2f4a32a314a0351503f750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:49.917281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.917977 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10deb520-17e0-4041-aeac-079b65774245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:49.919542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1143/2298] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:49 2022] DELETE /identity/v3/projects/60647f765b2f4a32a314a0351503f750 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:49.924635 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea336-3fb5-4df8-b948-4877875f19ac None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:49.925427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea336-3fb5-4df8-b948-4877875f19ac None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:49.925427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ea336-3fb5-4df8-b948-4877875f19ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:49.945528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d99ea336-3fb5-4df8-b948-4877875f19ac None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:49.946917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d99ea336-3fb5-4df8-b948-4877875f19ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:50.015085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1156/2299] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:50.023188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947fc7f-3184-436b-8b61-7e5c72149060 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.023828 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947fc7f-3184-436b-8b61-7e5c72149060 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.024094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947fc7f-3184-436b-8b61-7e5c72149060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.046264 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f947fc7f-3184-436b-8b61-7e5c72149060 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:50.047478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f947fc7f-3184-436b-8b61-7e5c72149060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:50.055554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1144/2300] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:50.060247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea336-3fb5-4df8-b948-4877875f19ac None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.070865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ea336-3fb5-4df8-b948-4877875f19ac None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.075590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.076198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.076496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.076764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.077418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:50.077716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.079331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.091436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1157/2301] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:03:50.096237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947fc7f-3184-436b-8b61-7e5c72149060 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.105701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947fc7f-3184-436b-8b61-7e5c72149060 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.111049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.111687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.112105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.112516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.113177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.115171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.148119 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1145/2302] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:50 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:50.153159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.167211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad0ecc-0089-40eb-8c67-8208139e804b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.240629 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.241256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.241540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.241815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.242487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.244644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.269465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1158/2303] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:50 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:50.275206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.287783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41c8a-6c01-4e78-a465-46e5279e943d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.293311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.293927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.294207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.294479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.295132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.296812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.307705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1146/2304] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:50.312073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.322923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51920444-b3be-44c1-bed9-7fd385cfc8de admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.328542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cf272-2f9f-4280-816d-1214a563e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.329199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf272-2f9f-4280-816d-1214a563e2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.329478 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf272-2f9f-4280-816d-1214a563e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.329757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cf272-2f9f-4280-816d-1214a563e2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.330417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cf272-2f9f-4280-816d-1214a563e2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.332070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cf272-2f9f-4280-816d-1214a563e2d7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.344870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1159/2305] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:50.349760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.361704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0ad63-5b65-4ebb-a583-2ac78efd0a99 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.368684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.369593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.369896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.370167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] PATH_INFO: `/v3/projects/4de11246307846fb8e938814c93a3f48/users/ee8de17a7cb84607b2df8168ab32340f/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.391473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de11246307846fb8e938814c93a3f48, user_id=ee8de17a7cb84607b2df8168ab32340f, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.394268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.409629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1147/2306] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:50 2022] PUT /identity/v3/projects/4de11246307846fb8e938814c93a3f48/users/ee8de17a7cb84607b2df8168ab32340f/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:50.420449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b8e3-882a-4964-849e-9a3e69a888b1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.420642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b8e3-882a-4964-849e-9a3e69a888b1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.420916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21b8e3-882a-4964-849e-9a3e69a888b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.455883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0a21b8e3-882a-4964-849e-9a3e69a888b1 None None] MFA Rules not processed for user `ee8de17a7cb84607b2df8168ab32340f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:50.455883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0a21b8e3-882a-4964-849e-9a3e69a888b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:50.512940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:50.525632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1160/2307] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:50.530474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.602389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9024e-b77f-4e56-8ea8-a2fa16105d39 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.608138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.608769 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.609048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.609326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.610079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.611952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.648897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1148/2308] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:50 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:50.650245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21b8e3-882a-4964-849e-9a3e69a888b1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.662936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21b8e3-882a-4964-849e-9a3e69a888b1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.725052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.725602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.725876 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.726142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.727332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.728832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.762233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1161/2309] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:50 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:50.763525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.774921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddfae8-bb39-4284-bdd6-723a1c513218 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.780708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.781532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.781907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.782248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.782972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.784748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.802332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1149/2310] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:50.803017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.813638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478f222-e067-417e-ad66-775ee8450776 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.818776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.819438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.819718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.819990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] PATH_INFO: `/v3/projects/f6dbcb9e2db7498baf47b29294ceafbb/users/32d8e405186b46f4ac224fc8ba7cde67/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.835702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dbcb9e2db7498baf47b29294ceafbb, user_id=32d8e405186b46f4ac224fc8ba7cde67, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.838496 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.854577 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1162/2311] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:50 2022] PUT /identity/v3/projects/f6dbcb9e2db7498baf47b29294ceafbb/users/32d8e405186b46f4ac224fc8ba7cde67/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:50.860140 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.875726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd719-6a5c-4c44-b600-f8473e6ff9f2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.933540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:50.948176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fef6d-3a3a-4767-8432-7ee246073174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.948842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fef6d-3a3a-4767-8432-7ee246073174 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.949121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fef6d-3a3a-4767-8432-7ee246073174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.949398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fef6d-3a3a-4767-8432-7ee246073174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.950062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fef6d-3a3a-4767-8432-7ee246073174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.951754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fef6d-3a3a-4767-8432-7ee246073174 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:50.963288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1150/2312] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:50.967949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:50.978730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbec599-461b-4ab2-b24d-768589d6ea12 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:50.984019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:50.984735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:50.985070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:50.985459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] PATH_INFO: `/v3/projects/f6dbcb9e2db7498baf47b29294ceafbb/users/32d8e405186b46f4ac224fc8ba7cde67/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:50.987675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dbcb9e2db7498baf47b29294ceafbb, user_id=32d8e405186b46f4ac224fc8ba7cde67, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:50.990363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:51.001830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1163/2313] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:50 2022] PUT /identity/v3/projects/f6dbcb9e2db7498baf47b29294ceafbb/users/32d8e405186b46f4ac224fc8ba7cde67/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:51.014475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35418acc-1ca4-4842-909a-d8662210ba9d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:51.014732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35418acc-1ca4-4842-909a-d8662210ba9d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:51.015059 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35418acc-1ca4-4842-909a-d8662210ba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:51.045317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-35418acc-1ca4-4842-909a-d8662210ba9d None None] MFA Rules not processed for user `32d8e405186b46f4ac224fc8ba7cde67`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:51.046537 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-35418acc-1ca4-4842-909a-d8662210ba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:51.122700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1151/2314] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:51.134477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:51.136887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:51.206779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618e6f-d5be-4640-8cf5-647fb1375a44 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:51.210836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519934c-0302-4339-b94e-0f61f3af6556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:51.211519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519934c-0302-4339-b94e-0f61f3af6556 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:51.211823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519934c-0302-4339-b94e-0f61f3af6556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:51.212096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519934c-0302-4339-b94e-0f61f3af6556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:51.230120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519934c-0302-4339-b94e-0f61f3af6556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:51.231545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519934c-0302-4339-b94e-0f61f3af6556 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:51.242626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1164/2315] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:03:52.762404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35418acc-1ca4-4842-909a-d8662210ba9d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:52.825448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35418acc-1ca4-4842-909a-d8662210ba9d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:52.830676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:52.831317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:52.831592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:52.831859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] PATH_INFO: `/v3/users/ee8de17a7cb84607b2df8168ab32340f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:52.833125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee8de17a7cb84607b2df8168ab32340f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:52.835110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:52.884583 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee8de17a7cb84607b2df8168ab32340f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:52.893064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee8de17a7cb84607b2df8168ab32340f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:52.893971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee8de17a7cb84607b2df8168ab32340f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:52.902001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee8de17a7cb84607b2df8168ab32340f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:52.914457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:52.919565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee8de17a7cb84607b2df8168ab32340f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:52.929117 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1152/2316] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:52 2022] DELETE /identity/v3/users/ee8de17a7cb84607b2df8168ab32340f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:52.933777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:52.934613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:52.934939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:52.958979 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:52.960231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:53.012264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:53.034062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1165/2317] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:53.043234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:53.046099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:53.125929 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc819be-5d48-4739-9bae-65cb24f41f6b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:53.129882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:53.130639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:53.130911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:53.131174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:53.140765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:53.142300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:53.155020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1153/2318] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:53.454038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:53.456270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:53.478411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f4ab7-6ead-4459-874d-b1d43b1d672a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:53.483360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:53.483998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:53.484276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:53.484583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] PATH_INFO: `/v3/projects/4de11246307846fb8e938814c93a3f48` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:53.485729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de11246307846fb8e938814c93a3f48)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:53.487641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:53.546628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de11246307846fb8e938814c93a3f48', 'request_id': 'req-e9039b6d-70ba-4b74-93f6-04b26b943df3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.559914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de11246307846fb8e938814c93a3f48', 'request_id': 'req-e9039b6d-70ba-4b74-93f6-04b26b943df3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.568872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de11246307846fb8e938814c93a3f48', 'request_id': 'req-e9039b6d-70ba-4b74-93f6-04b26b943df3'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.585280 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] The token cache is being invalidate because project 4de11246307846fb8e938814c93a3f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:53.585706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.587099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.589248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1166/2319] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:53 2022] DELETE /identity/v3/projects/4de11246307846fb8e938814c93a3f48 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:53.594135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:53.606142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886d953-2153-4265-bd99-190d75469f63 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:53.701011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:53.702357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:53.702715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:53.703145 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] PATH_INFO: `/v3/users/32d8e405186b46f4ac224fc8ba7cde67` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:53.704548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d8e405186b46f4ac224fc8ba7cde67)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:53.708172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:53.774324 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d8e405186b46f4ac224fc8ba7cde67'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.783820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d8e405186b46f4ac224fc8ba7cde67'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.784746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d8e405186b46f4ac224fc8ba7cde67'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.795441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d8e405186b46f4ac224fc8ba7cde67'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.811398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d6fd8092-d2f2-411a-bc1a-7563a3d1525c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d8e405186b46f4ac224fc8ba7cde67'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:53.818343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1154/2320] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:53 2022] DELETE /identity/v3/users/32d8e405186b46f4ac224fc8ba7cde67 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:54.159506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:54.161026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:54.179452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039b6d-70ba-4b74-93f6-04b26b943df3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:54.248771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:54.249483 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.249857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.250176 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] PATH_INFO: `/v3/projects/f6dbcb9e2db7498baf47b29294ceafbb` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.251504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6dbcb9e2db7498baf47b29294ceafbb)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:54.253912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:54.318367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6dbcb9e2db7498baf47b29294ceafbb', 'request_id': 'req-546b2f44-38b9-4949-a4ed-215f54b7501f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:54.329282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6dbcb9e2db7498baf47b29294ceafbb', 'request_id': 'req-546b2f44-38b9-4949-a4ed-215f54b7501f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:54.338141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6dbcb9e2db7498baf47b29294ceafbb', 'request_id': 'req-546b2f44-38b9-4949-a4ed-215f54b7501f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:54.356818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] The token cache is being invalidate because project f6dbcb9e2db7498baf47b29294ceafbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:54.357161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:54.357782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-546b2f44-38b9-4949-a4ed-215f54b7501f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:54.359747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1167/2321] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:54 2022] DELETE /identity/v3/projects/f6dbcb9e2db7498baf47b29294ceafbb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:54.365198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aabc0a-549f-4a8b-b74a-2b0dbd949570 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.365353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aabc0a-549f-4a8b-b74a-2b0dbd949570 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.365628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aabc0a-549f-4a8b-b74a-2b0dbd949570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.391256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-14aabc0a-549f-4a8b-b74a-2b0dbd949570 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:54.392531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14aabc0a-549f-4a8b-b74a-2b0dbd949570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:54.405927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:54.469313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1155/2322] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:54.477675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802aefd-547f-4fec-ab21-f15cd0483033 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.480480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802aefd-547f-4fec-ab21-f15cd0483033 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.480480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802aefd-547f-4fec-ab21-f15cd0483033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.486489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:54.506597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c802aefd-547f-4fec-ab21-f15cd0483033 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:54.508137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c802aefd-547f-4fec-ab21-f15cd0483033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:54.517123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1168/2323] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:54.521115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aabc0a-549f-4a8b-b74a-2b0dbd949570 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:54.531638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aabc0a-549f-4a8b-b74a-2b0dbd949570 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:54.536649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:54.537143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.537419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.537686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.538393 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:03:54.538797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:54.540271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:54.557690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1156/2324] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:03:54.558863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802aefd-547f-4fec-ab21-f15cd0483033 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:54.569459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802aefd-547f-4fec-ab21-f15cd0483033 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:54.574559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:54.575191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.575466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.575728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.576358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:54.578297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:54.610405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1169/2325] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:54 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:54.611663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:54.622139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920f954-695a-43cb-aba8-224a84490d75 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:54.682664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:54.683280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.683555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.683812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.684477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:54.686452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:54.712439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1157/2326] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:54 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:54.716906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:54.728017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab69b78-7032-40ee-b11c-57e45ccd66aa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:54.733641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:54.734372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.734768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.735094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.736248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:54.737472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:54.749335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1170/2327] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:54.754996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:54.769958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6d58c-0e18-411b-903f-f6533ab244f5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:54.776410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2f1db-db38-4147-af90-6c40ffd4c084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:54.777175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2f1db-db38-4147-af90-6c40ffd4c084 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.777551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2f1db-db38-4147-af90-6c40ffd4c084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.777907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2f1db-db38-4147-af90-6c40ffd4c084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.778663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2f1db-db38-4147-af90-6c40ffd4c084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:54.780398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2f1db-db38-4147-af90-6c40ffd4c084 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:54.793617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1158/2328] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:54.797910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:54.808503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefde88-c6d2-4860-8eaa-1ddfee144804 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:54.813639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:54.814350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.814632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.814909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] PATH_INFO: `/v3/projects/b39a5beed82a4943ac5e9e1be00c8742/users/0132e78bdc154e68836d89834e93e26e/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.830878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b39a5beed82a4943ac5e9e1be00c8742, user_id=0132e78bdc154e68836d89834e93e26e, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:54.834484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:54.853498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1171/2329] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:54 2022] PUT /identity/v3/projects/b39a5beed82a4943ac5e9e1be00c8742/users/0132e78bdc154e68836d89834e93e26e/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:54.865440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f49e0-e1af-449c-9127-fcb5c4342e26 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:54.865831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f49e0-e1af-449c-9127-fcb5c4342e26 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:54.866024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f49e0-e1af-449c-9127-fcb5c4342e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:54.898318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8f0f49e0-e1af-449c-9127-fcb5c4342e26 None None] MFA Rules not processed for user `0132e78bdc154e68836d89834e93e26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:54.899686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8f0f49e0-e1af-449c-9127-fcb5c4342e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:54.968832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1159/2330] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:03:54.973351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.049570 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23465f63-8e8c-4b2e-bfa0-39fcccd7f467 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.055364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.055967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.056242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.056553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.057396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.059508 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.098995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1172/2331] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:03:54 2022] POST /identity/v3/projects => generated 399 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:03:55.100660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f49e0-e1af-449c-9127-fcb5c4342e26 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.113137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f49e0-e1af-449c-9127-fcb5c4342e26 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.124772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:55.186183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.186805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.187149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.187422 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.188077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.190477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.215252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1160/2332] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:03:55 2022] POST /identity/v3/users => generated 306 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:03:55.220314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.231404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1c58-4a9c-4138-9539-92fa0ed0922d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.236646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.237267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.237548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.237821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.238470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.240168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.252845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1173/2333] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:03:55.257525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.269698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a891f9-4c1c-4829-9e92-64dd05c5e973 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.274700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.275438 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.275715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.275991 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] PATH_INFO: `/v3/projects/bb9233a968df4094916aa3a3d239ebcc/users/7e4cca2b71364674b45988d48ef36280/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.291701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9233a968df4094916aa3a3d239ebcc, user_id=7e4cca2b71364674b45988d48ef36280, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.294421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.310448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1161/2334] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:55 2022] PUT /identity/v3/projects/bb9233a968df4094916aa3a3d239ebcc/users/7e4cca2b71364674b45988d48ef36280/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:03:55.314561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.338046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d717ef1-00ba-4650-8ed4-edb6dc9dc4d8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.412974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97babf9-b437-4b0e-8f44-378bc9a2f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.413502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97babf9-b437-4b0e-8f44-378bc9a2f5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.413784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97babf9-b437-4b0e-8f44-378bc9a2f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.414553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97babf9-b437-4b0e-8f44-378bc9a2f5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.414733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97babf9-b437-4b0e-8f44-378bc9a2f5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.417250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97babf9-b437-4b0e-8f44-378bc9a2f5e1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.428824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1174/2335] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:03:55.433249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.444578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed826481-edca-4699-b797-bfb4a83825f4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.450556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.451295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.451713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.451896 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] PATH_INFO: `/v3/projects/bb9233a968df4094916aa3a3d239ebcc/users/7e4cca2b71364674b45988d48ef36280/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.454084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9233a968df4094916aa3a3d239ebcc, user_id=7e4cca2b71364674b45988d48ef36280, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.456756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.470261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1162/2336] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:03:55 2022] PUT /identity/v3/projects/bb9233a968df4094916aa3a3d239ebcc/users/7e4cca2b71364674b45988d48ef36280/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:55.480795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f1ff6-822b-4c15-9f17-444884a287e8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.481195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f1ff6-822b-4c15-9f17-444884a287e8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.482306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f1ff6-822b-4c15-9f17-444884a287e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.511003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:55.523407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-181f1ff6-822b-4c15-9f17-444884a287e8 None None] MFA Rules not processed for user `7e4cca2b71364674b45988d48ef36280`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:55.524788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-181f1ff6-822b-4c15-9f17-444884a287e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:55.614122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1175/2337] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:55.625114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.626935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:55.693571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7d4bc-1ca6-4391-a294-fe648d3be876 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.697535 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.698151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.698468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.698736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.710027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.711760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.722229 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1163/2338] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:55.803371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f1ff6-822b-4c15-9f17-444884a287e8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.813764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f1ff6-822b-4c15-9f17-444884a287e8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:55.817359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba208e9-3853-4967-9e2c-fdcfb49bcae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:55.817966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba208e9-3853-4967-9e2c-fdcfb49bcae5 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:55.818237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba208e9-3853-4967-9e2c-fdcfb49bcae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:55.818504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba208e9-3853-4967-9e2c-fdcfb49bcae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:55.876613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba208e9-3853-4967-9e2c-fdcfb49bcae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:55.878665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba208e9-3853-4967-9e2c-fdcfb49bcae5 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:55.888230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1176/2339] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:55.928308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:55.930073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:56.014516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4603c82-5376-4a67-93a4-f684e4e2b65f service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:56.021441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:56.022143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:56.022492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:56.022961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] PATH_INFO: `/v3/users/dd9d4069961a47c78351786a7aef4ad4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:56.024261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9d4069961a47c78351786a7aef4ad4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:56.026663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:56.038074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:56.092322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9d4069961a47c78351786a7aef4ad4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.103153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9d4069961a47c78351786a7aef4ad4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.103989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9d4069961a47c78351786a7aef4ad4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.119143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9d4069961a47c78351786a7aef4ad4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.135971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9d4069961a47c78351786a7aef4ad4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.143030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1164/2340] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:03:55 2022] DELETE /identity/v3/users/dd9d4069961a47c78351786a7aef4ad4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:03:56.148124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda96992-58d9-4f2d-bbde-faf7323e87a7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:56.148457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda96992-58d9-4f2d-bbde-faf7323e87a7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:56.148719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda96992-58d9-4f2d-bbde-faf7323e87a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:56.194434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bda96992-58d9-4f2d-bbde-faf7323e87a7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:56.194434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bda96992-58d9-4f2d-bbde-faf7323e87a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:56.289092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1177/2341] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:56.297793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:56.299473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:56.363698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308072e-cb8e-4364-a962-1ca508231efb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:56.367307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:56.367957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:56.368236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:56.368543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:56.378394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:56.379889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:56.389992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1165/2342] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:03:56.591913 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:56.592208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:56.592538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:56.612657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] MFA Rules not processed for user `60800cf2774e409d9c1b6f4992669991`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:03:56.614057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:56.651885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:03:56.691777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1178/2343] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:03:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:03:56.698365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:03:56.699836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:03:56.744538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea02082-a580-449e-9192-46cdc5871c73 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:03:56.750233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:03:56.750878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:03:56.751150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:03:56.751419 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] PATH_INFO: `/v3/projects/77667d7dcad04b1aaaae178536cdf127` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:03:56.753047 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] RBAC: Authorizing `identity:delete_project(project_id=77667d7dcad04b1aaaae178536cdf127)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:03:56.755074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:03:56.820970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77667d7dcad04b1aaaae178536cdf127', 'request_id': 'req-4040cf11-a6dd-43b2-bae2-a0476d08141e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.832091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77667d7dcad04b1aaaae178536cdf127', 'request_id': 'req-4040cf11-a6dd-43b2-bae2-a0476d08141e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.840275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77667d7dcad04b1aaaae178536cdf127', 'request_id': 'req-4040cf11-a6dd-43b2-bae2-a0476d08141e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.852459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] The token cache is being invalidate because project 77667d7dcad04b1aaaae178536cdf127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:03:56.852459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.853301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4040cf11-a6dd-43b2-bae2-a0476d08141e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:03:56.854877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1166/2344] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:03:56 2022] DELETE /identity/v3/projects/77667d7dcad04b1aaaae178536cdf127 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:00.180669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:00.184087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:00.205497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da30ba-2ba3-4b64-89f0-012e6a3630f7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:00.274090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:00.274930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:00.275317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:00.275689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] PATH_INFO: `/v3/users/0132e78bdc154e68836d89834e93e26e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:00.276986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0132e78bdc154e68836d89834e93e26e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:00.279074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:00.349290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0132e78bdc154e68836d89834e93e26e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:00.359471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0132e78bdc154e68836d89834e93e26e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:00.360412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0132e78bdc154e68836d89834e93e26e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:00.374533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0132e78bdc154e68836d89834e93e26e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:00.399068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0132e78bdc154e68836d89834e93e26e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:00.412254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1179/2345] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:04:00 2022] DELETE /identity/v3/users/0132e78bdc154e68836d89834e93e26e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:00.412489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:00.412995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:00.412995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:00.435494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:00.437052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:00.444406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:00.558322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1167/2346] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:04:00.602795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:00.604869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:00.694589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03d287-f836-4664-9010-bba1e83794ef admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:00.716264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:00.716264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:00.716264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:00.716264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:00.738076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:00.739728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:00.757924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:00.767441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1180/2347] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:04:01.310286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:01.311760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:01.331595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf697dc-ba75-4adc-a298-90010862d3d1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:01.342234 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:01.343348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:01.343809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:01.344407 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] PATH_INFO: `/v3/projects/b39a5beed82a4943ac5e9e1be00c8742` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:01.346500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b39a5beed82a4943ac5e9e1be00c8742)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:01.349741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:01.434031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b39a5beed82a4943ac5e9e1be00c8742', 'request_id': 'req-be26ed58-fd10-42a1-86d5-8a9e153531da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.446477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b39a5beed82a4943ac5e9e1be00c8742', 'request_id': 'req-be26ed58-fd10-42a1-86d5-8a9e153531da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.460482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b39a5beed82a4943ac5e9e1be00c8742', 'request_id': 'req-be26ed58-fd10-42a1-86d5-8a9e153531da'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.486032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] The token cache is being invalidate because project b39a5beed82a4943ac5e9e1be00c8742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:04:01.486522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.495554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-be26ed58-fd10-42a1-86d5-8a9e153531da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.507104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:01.507569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1168/2348] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:04:01 2022] DELETE /identity/v3/projects/b39a5beed82a4943ac5e9e1be00c8742 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:01.534248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78304e-cbe3-41e3-a34b-6a2e88416b14 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:01.630245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:01.631264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:01.631707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:01.633315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] PATH_INFO: `/v3/users/7e4cca2b71364674b45988d48ef36280` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:01.635091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4cca2b71364674b45988d48ef36280)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:01.637360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:01.749078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4cca2b71364674b45988d48ef36280'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.769704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4cca2b71364674b45988d48ef36280'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.770325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4cca2b71364674b45988d48ef36280'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.795519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4cca2b71364674b45988d48ef36280'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.830204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:01.830486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:01.830758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:01.836789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4cca2b71364674b45988d48ef36280'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:01.852237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1181/2349] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:04:01 2022] DELETE /identity/v3/users/7e4cca2b71364674b45988d48ef36280 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:01.858986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:01.860379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:01.933223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1169/2350] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:01.983770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:01.985401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:02.077140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895e6d-5a1e-4750-98af-b9b851f3e154 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:02.083974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b0ccc-38bb-46eb-a0c6-d85f2c361ce8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:02.084622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0ccc-38bb-46eb-a0c6-d85f2c361ce8 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:02.084960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0ccc-38bb-46eb-a0c6-d85f2c361ce8 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:02.088676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b0ccc-38bb-46eb-a0c6-d85f2c361ce8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:02.099003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b0ccc-38bb-46eb-a0c6-d85f2c361ce8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:02.100071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b0ccc-38bb-46eb-a0c6-d85f2c361ce8 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:02.110249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1182/2351] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:04:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:04:02.615171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:02.617937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:02.710984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:02.748217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862a702-a7f0-4ce7-93f3-793168c37842 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:02.761439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:02.764571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:02.764571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:02.764571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] PATH_INFO: `/v3/projects/bb9233a968df4094916aa3a3d239ebcc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:02.766220 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb9233a968df4094916aa3a3d239ebcc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:02.769413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:02.859012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb9233a968df4094916aa3a3d239ebcc', 'request_id': 'req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:02.869376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb9233a968df4094916aa3a3d239ebcc', 'request_id': 'req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:02.877630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb9233a968df4094916aa3a3d239ebcc', 'request_id': 'req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:02.892821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] The token cache is being invalidate because project bb9233a968df4094916aa3a3d239ebcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:04:02.893110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:02.893761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f23fc23b-fdfb-4b5b-bd87-3d2d9a85c311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:02.895542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1170/2352] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:04:02 2022] DELETE /identity/v3/projects/bb9233a968df4094916aa3a3d239ebcc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:02.900471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aeea4-6829-49c2-ae4d-727122099ecc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:02.900986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aeea4-6829-49c2-ae4d-727122099ecc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:02.900986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aeea4-6829-49c2-ae4d-727122099ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:02.928409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d18aeea4-6829-49c2-ae4d-727122099ecc None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:02.930787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d18aeea4-6829-49c2-ae4d-727122099ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:02.939055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:03.039745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1183/2353] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:03.043542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4c57-ae01-4cf4-ac4e-8c13f744296e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.043542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4c57-ae01-4cf4-ac4e-8c13f744296e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.043542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4c57-ae01-4cf4-ac4e-8c13f744296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.077639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e07c4c57-ae01-4cf4-ac4e-8c13f744296e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:03.078887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e07c4c57-ae01-4cf4-ac4e-8c13f744296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:03.094015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1171/2354] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:04:03.099981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aeea4-6829-49c2-ae4d-727122099ecc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:03.110670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aeea4-6829-49c2-ae4d-727122099ecc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:03.115520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:03.116385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.116790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.117281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.118017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:04:03.118384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:03.120077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:03.136514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1184/2355] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:04:03.139449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4c57-ae01-4cf4-ac4e-8c13f744296e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:03.149771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4c57-ae01-4cf4-ac4e-8c13f744296e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:03.154638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:03.155356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.155746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.156113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.156884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:03.159196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:03.194162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1172/2356] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:04:03 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:04:03.200252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:03.230453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e1963-483b-4f51-951f-5ca9c4a883d6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:03.341500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:03.342612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.343653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.344092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.344926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:03.348938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:03.381126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1185/2357] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:04:03 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:04:03.383061 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:03.397037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa0da9-a1f0-40de-8664-c4b4fbe9a2aa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:03.411518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:03.412171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.412457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.412820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.413571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:03.415236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:03.436832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1173/2358] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:04:03.440652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:03.455009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9902e-9b32-461d-9332-a8287f7ed3f3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:03.477257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26378f18-22ac-4de1-b49d-6ae422a062b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:03.478174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26378f18-22ac-4de1-b49d-6ae422a062b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.478174 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26378f18-22ac-4de1-b49d-6ae422a062b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.478590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26378f18-22ac-4de1-b49d-6ae422a062b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.479463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26378f18-22ac-4de1-b49d-6ae422a062b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:03.480888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26378f18-22ac-4de1-b49d-6ae422a062b3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:03.498476 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1186/2359] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:04:03.503163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:03.521010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cec18f-58b1-444d-a593-6d160c7f2aea admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:03.526261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:03.526928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.527205 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.527474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] PATH_INFO: `/v3/projects/e9e019cd8d914aeea21db874c4976b1e/users/65db2e628b794dd6a38cf1ecff49c202/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.547090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e019cd8d914aeea21db874c4976b1e, user_id=65db2e628b794dd6a38cf1ecff49c202, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:03.551041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:03.580598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1174/2360] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:04:03 2022] PUT /identity/v3/projects/e9e019cd8d914aeea21db874c4976b1e/users/65db2e628b794dd6a38cf1ecff49c202/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:04:03.582199 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a2db-161c-4b54-9ccf-a36838665383 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.582553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a2db-161c-4b54-9ccf-a36838665383 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.583012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a2db-161c-4b54-9ccf-a36838665383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.635820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-d4f7a2db-161c-4b54-9ccf-a36838665383 None None] MFA Rules not processed for user `65db2e628b794dd6a38cf1ecff49c202`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:03.637451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d4f7a2db-161c-4b54-9ccf-a36838665383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:03.718291 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1187/2361] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:04:03.727346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:03.729871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:03.866946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968e698-f563-4bee-b073-fe116e4521ec admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:03.878124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f89b9-4ac5-483d-9762-a4d34f5465d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:03.878823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f89b9-4ac5-483d-9762-a4d34f5465d0 service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:03.879248 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f89b9-4ac5-483d-9762-a4d34f5465d0 service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:03.879531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f89b9-4ac5-483d-9762-a4d34f5465d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:03.895169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f89b9-4ac5-483d-9762-a4d34f5465d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:03.901848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f89b9-4ac5-483d-9762-a4d34f5465d0 service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:03.913695 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1175/2362] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:04:13.710557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:13.908591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:17.516396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92172cf0-c620-45fe-9a0a-cd2e0b7eb77f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:17.516993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92172cf0-c620-45fe-9a0a-cd2e0b7eb77f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:17.516993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92172cf0-c620-45fe-9a0a-cd2e0b7eb77f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:17.518324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1188/2363] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:04:17.524787 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9843f6-6faa-4cb7-a24d-786f7448ff94 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:17.525294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9843f6-6faa-4cb7-a24d-786f7448ff94 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:17.525402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9843f6-6faa-4cb7-a24d-786f7448ff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:17.545822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7a9843f6-6faa-4cb7-a24d-786f7448ff94 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:17.547063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7a9843f6-6faa-4cb7-a24d-786f7448ff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:17.626133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1176/2364] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:04:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:17.642751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92172cf0-c620-45fe-9a0a-cd2e0b7eb77f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:17.645120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-92172cf0-c620-45fe-9a0a-cd2e0b7eb77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:17.658947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92172cf0-c620-45fe-9a0a-cd2e0b7eb77f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:17.662564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd88e9-abb6-4ae8-ad4d-275a08e02664 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:17.663160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd88e9-abb6-4ae8-ad4d-275a08e02664 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:17.663433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd88e9-abb6-4ae8-ad4d-275a08e02664 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:17.663696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd88e9-abb6-4ae8-ad4d-275a08e02664 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:17.673922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd88e9-abb6-4ae8-ad4d-275a08e02664 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:17.675329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd88e9-abb6-4ae8-ad4d-275a08e02664 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:17.686077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1189/2365] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:04:17.760724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185b9d-0bd4-4496-8381-6d5d261076f1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:17.761010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185b9d-0bd4-4496-8381-6d5d261076f1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:17.761282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185b9d-0bd4-4496-8381-6d5d261076f1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:17.762663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1177/2366] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:04:17.767880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d39f-3d6c-4246-9780-3c58784efad2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:17.768164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d39f-3d6c-4246-9780-3c58784efad2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:17.768500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9d39f-3d6c-4246-9780-3c58784efad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:17.790986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e2b9d39f-3d6c-4246-9780-3c58784efad2 None None] MFA Rules not processed for user `a4b10367002d4f64a82b3e9eca00ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:17.792204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e2b9d39f-3d6c-4246-9780-3c58784efad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:17.846734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1190/2367] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:04:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:17.870627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7185b9d-0bd4-4496-8381-6d5d261076f1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:17.873389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d7185b9d-0bd4-4496-8381-6d5d261076f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:17.888285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7185b9d-0bd4-4496-8381-6d5d261076f1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:17.892389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eb5af-151d-4b18-add9-bdd0b866d9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:17.893115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb5af-151d-4b18-add9-bdd0b866d9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:17.893541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb5af-151d-4b18-add9-bdd0b866d9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:17.893825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb5af-151d-4b18-add9-bdd0b866d9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:17.902065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eb5af-151d-4b18-add9-bdd0b866d9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:17.903544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eb5af-151d-4b18-add9-bdd0b866d9a2 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:17.912078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1178/2368] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:04:18.031539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705269b-f82b-4fc3-a622-9f60e3dbd6ce None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:18.031857 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705269b-f82b-4fc3-a622-9f60e3dbd6ce None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:18.032135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705269b-f82b-4fc3-a622-9f60e3dbd6ce None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:18.033550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1191/2369] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:04:18.039090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad4a6-4fa3-4952-baa5-bc59525edd9a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:18.039378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad4a6-4fa3-4952-baa5-bc59525edd9a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:18.039653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ad4a6-4fa3-4952-baa5-bc59525edd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:18.064582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fe6ad4a6-4fa3-4952-baa5-bc59525edd9a None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:18.065829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fe6ad4a6-4fa3-4952-baa5-bc59525edd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:18.075779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1179/2370] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:04:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:18.088534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705269b-f82b-4fc3-a622-9f60e3dbd6ce None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:18.090327 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a705269b-f82b-4fc3-a622-9f60e3dbd6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:18.104359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705269b-f82b-4fc3-a622-9f60e3dbd6ce None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:18.107971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c267a-cef3-4489-9849-d88041551245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:18.108612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c267a-cef3-4489-9849-d88041551245 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:18.108917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c267a-cef3-4489-9849-d88041551245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:18.109188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c267a-cef3-4489-9849-d88041551245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:18.120986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c267a-cef3-4489-9849-d88041551245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:18.122516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c267a-cef3-4489-9849-d88041551245 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:18.134335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1192/2371] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:04:19.548218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ad4a6-4fa3-4952-baa5-bc59525edd9a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:19.560024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ad4a6-4fa3-4952-baa5-bc59525edd9a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:19.563845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:19.564768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:19.565205 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:19.565526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:19.567723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:19.674177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:19.675953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:19.706173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1180/2372] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:04:21.896801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300729-b338-4ab2-9fbc-4a313ef74815 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:21.897704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300729-b338-4ab2-9fbc-4a313ef74815 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:21.898127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300729-b338-4ab2-9fbc-4a313ef74815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:21.930821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-14300729-b338-4ab2-9fbc-4a313ef74815 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:21.935471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14300729-b338-4ab2-9fbc-4a313ef74815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:22.053731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1193/2373] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:04:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:22.061414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:22.063763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:22.210865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28c340-8fea-4006-bf81-f1f90d329058 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:22.239827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c02b9-1b2c-4267-929a-cdfc3d01026a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:22.242127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c02b9-1b2c-4267-929a-cdfc3d01026a service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:22.242471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c02b9-1b2c-4267-929a-cdfc3d01026a service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:22.242710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c02b9-1b2c-4267-929a-cdfc3d01026a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:22.260307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:22.266384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c02b9-1b2c-4267-929a-cdfc3d01026a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:22.268083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c02b9-1b2c-4267-929a-cdfc3d01026a service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:22.278110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1181/2374] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:04:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 18:04:32.031248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=60 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:32.273912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:38.189621 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14300729-b338-4ab2-9fbc-4a313ef74815 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:38.264529 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14300729-b338-4ab2-9fbc-4a313ef74815 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:38.272773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:38.273431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:38.273708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:38.273976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] PATH_INFO: `/v3/users/65db2e628b794dd6a38cf1ecff49c202` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:38.275205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65db2e628b794dd6a38cf1ecff49c202)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:38.277220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:38.330482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65db2e628b794dd6a38cf1ecff49c202'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.338494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65db2e628b794dd6a38cf1ecff49c202'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.339225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65db2e628b794dd6a38cf1ecff49c202'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.346179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65db2e628b794dd6a38cf1ecff49c202'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.358490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65db2e628b794dd6a38cf1ecff49c202'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.364456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1194/2375] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:04:38 2022] DELETE /identity/v3/users/65db2e628b794dd6a38cf1ecff49c202 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:38.369387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:38.370441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:38.370672 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:38.391197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:38.392477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:38.451912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1182/2376] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:38.461885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:38.463308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:38.521735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fced2-a432-435a-9cfa-1097d67a95b2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:38.526121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f34d1-fee8-4e0b-9f74-fa5060c2436a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:38.526813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f34d1-fee8-4e0b-9f74-fa5060c2436a service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:38.527093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f34d1-fee8-4e0b-9f74-fa5060c2436a service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:38.527395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f34d1-fee8-4e0b-9f74-fa5060c2436a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:38.538705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f34d1-fee8-4e0b-9f74-fa5060c2436a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:38.540134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f34d1-fee8-4e0b-9f74-fa5060c2436a service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:38.550481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1195/2377] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:04:38.834280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:38.836204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:38.850601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82f67c-5ea7-4e5d-9137-252375b87aec None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:38.856706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:38.857391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:38.857665 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:38.857939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] PATH_INFO: `/v3/projects/e9e019cd8d914aeea21db874c4976b1e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:38.859192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e019cd8d914aeea21db874c4976b1e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:38.861299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:38.909893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e019cd8d914aeea21db874c4976b1e', 'request_id': 'req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.918310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e019cd8d914aeea21db874c4976b1e', 'request_id': 'req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.925260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e019cd8d914aeea21db874c4976b1e', 'request_id': 'req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.935846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] The token cache is being invalidate because project e9e019cd8d914aeea21db874c4976b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:04:38.936149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.936772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6f082a13-6cad-4b1a-adc1-35cbb47fe03c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:04:38.938257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1183/2378] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:04:38 2022] DELETE /identity/v3/projects/e9e019cd8d914aeea21db874c4976b1e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:38.944157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd4315-4568-44d9-86a8-267678cbf2c9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:38.944482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd4315-4568-44d9-86a8-267678cbf2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:38.944754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd4315-4568-44d9-86a8-267678cbf2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:38.964114 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8cfd4315-4568-44d9-86a8-267678cbf2c9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:38.965339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8cfd4315-4568-44d9-86a8-267678cbf2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:38.984376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:39.029923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1196/2379] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:39.038716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f45b-85e6-4492-8cc0-942540a29814 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.039059 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f45b-85e6-4492-8cc0-942540a29814 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.039441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9f45b-85e6-4492-8cc0-942540a29814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.068528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-61c9f45b-85e6-4492-8cc0-942540a29814 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:39.069726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-61c9f45b-85e6-4492-8cc0-942540a29814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:39.079008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1184/2380] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:39.083356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd4315-4568-44d9-86a8-267678cbf2c9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:39.093814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd4315-4568-44d9-86a8-267678cbf2c9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:39.099777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:39.100402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.100719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.100986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.101625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:04:39.101926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:39.103502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:39.115488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1197/2381] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:04:39.120120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f45b-85e6-4492-8cc0-942540a29814 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:39.130179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9f45b-85e6-4492-8cc0-942540a29814 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:39.134985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:39.135605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.135939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.136214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.136969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:39.138886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:39.167688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1185/2382] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:04:39 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:04:39.171907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:39.182626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c73f61-16a5-40e0-8f88-d228f83b82f3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:39.243569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:39.244170 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.244400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.244726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.245361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:39.247421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:39.269861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1198/2383] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:04:39 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:04:39.274008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:39.284209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b440c-f783-465f-83d8-b7b33fb148b3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:39.288921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:39.289507 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.289782 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.290048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.290697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:39.292312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:39.302156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1186/2384] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:04:39.305969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:39.316091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e97b3-40b4-4eb2-a0c6-b24c9eaf7b28 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:39.320951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047e60a-15f4-4501-8c81-2caeca45513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:39.321542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047e60a-15f4-4501-8c81-2caeca45513f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.321820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047e60a-15f4-4501-8c81-2caeca45513f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.322087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047e60a-15f4-4501-8c81-2caeca45513f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.322728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047e60a-15f4-4501-8c81-2caeca45513f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:39.324324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9047e60a-15f4-4501-8c81-2caeca45513f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:39.335160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1199/2385] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:04:39.339533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:39.349939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4e5d7-a402-4cbe-bc0f-8e4e854768bd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:39.355367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:39.356023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.356303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.356631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] PATH_INFO: `/v3/projects/eaea892229d5459e94fbcd3dcfe11f33/users/5f41e1bd709b4550931d64e07a6a1482/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.371363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaea892229d5459e94fbcd3dcfe11f33, user_id=5f41e1bd709b4550931d64e07a6a1482, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:39.374094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:39.388011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1187/2386] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:04:39 2022] PUT /identity/v3/projects/eaea892229d5459e94fbcd3dcfe11f33/users/5f41e1bd709b4550931d64e07a6a1482/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:04:39.399397 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c508-d7f5-453c-be4f-8b05c3ff56d1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.399688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c508-d7f5-453c-be4f-8b05c3ff56d1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.399967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c508-d7f5-453c-be4f-8b05c3ff56d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.429111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7863c508-d7f5-453c-be4f-8b05c3ff56d1 None None] MFA Rules not processed for user `5f41e1bd709b4550931d64e07a6a1482`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:39.430475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7863c508-d7f5-453c-be4f-8b05c3ff56d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:39.495235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1200/2387] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:04:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:39.507533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:39.509613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:39.533054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:39.570380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868b8d7-1406-4da7-8cee-cad6ad6851e3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:39.574391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:39.574993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:39.575262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:39.575530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:39.583997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:39.585460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:39.594118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1188/2388] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:04:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:04:46.885640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366db6a-d896-46b5-8d92-329c79bb44ab None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:46.886160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366db6a-d896-46b5-8d92-329c79bb44ab None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:46.886160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366db6a-d896-46b5-8d92-329c79bb44ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:46.906848 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-4366db6a-d896-46b5-8d92-329c79bb44ab None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:04:46.908059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4366db6a-d896-46b5-8d92-329c79bb44ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:46.920646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:46.964654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1201/2389] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 13 18:04:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:04:46.977662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:04:46.979232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:04:47.056871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dc075-46af-4d89-b9ec-c00a6e04b00e service glance] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:04:47.062217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0d870-a37a-41dd-bbb6-578351c5de5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:04:47.062819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0d870-a37a-41dd-bbb6-578351c5de5b service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:04:47.063111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0d870-a37a-41dd-bbb6-578351c5de5b service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:04:47.063374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0d870-a37a-41dd-bbb6-578351c5de5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:04:47.072097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0d870-a37a-41dd-bbb6-578351c5de5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:04:47.073779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0d870-a37a-41dd-bbb6-578351c5de5b service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:04:47.082429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1189/2390] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:04:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:04:56.957482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=17 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:04:57.078555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=49 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:07.596470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366db6a-d896-46b5-8d92-329c79bb44ab None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:07.664288 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366db6a-d896-46b5-8d92-329c79bb44ab None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:07.669318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:07.669958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:07.670238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:07.670538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] PATH_INFO: `/v3/users/5f41e1bd709b4550931d64e07a6a1482` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:07.671689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f41e1bd709b4550931d64e07a6a1482)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:07.673650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:07.724187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f41e1bd709b4550931d64e07a6a1482'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:07.731416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f41e1bd709b4550931d64e07a6a1482'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:07.732076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f41e1bd709b4550931d64e07a6a1482'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:07.739242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f41e1bd709b4550931d64e07a6a1482'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:07.753844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f41e1bd709b4550931d64e07a6a1482'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:07.760572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1202/2391] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:05:07 2022] DELETE /identity/v3/users/5f41e1bd709b4550931d64e07a6a1482 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:07.765279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:07.765726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:07.765837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:07.786337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:07.787558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:07.851794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1190/2392] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:07.862606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:07.864133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:07.932136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622513dd-11e7-4616-a64c-9bf8ef28d675 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:07.936382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7dbd8-7979-47d6-9511-9796e248b1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:07.937029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7dbd8-7979-47d6-9511-9796e248b1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:07.937312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7dbd8-7979-47d6-9511-9796e248b1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:07.937581 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7dbd8-7979-47d6-9511-9796e248b1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:07.948281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7dbd8-7979-47d6-9511-9796e248b1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:07.949762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7dbd8-7979-47d6-9511-9796e248b1e0 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:07.964618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1203/2393] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:05:08.241335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.242811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:08.255228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a93a2-a87b-4bcf-9368-4a8284cd6fad None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.260657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.261284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.261560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.261828 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] PATH_INFO: `/v3/projects/eaea892229d5459e94fbcd3dcfe11f33` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.263044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaea892229d5459e94fbcd3dcfe11f33)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.265007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:08.315706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaea892229d5459e94fbcd3dcfe11f33', 'request_id': 'req-47bb730c-d4c1-47a7-9f51-44d6727b0251'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:08.324602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaea892229d5459e94fbcd3dcfe11f33', 'request_id': 'req-47bb730c-d4c1-47a7-9f51-44d6727b0251'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:08.331030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaea892229d5459e94fbcd3dcfe11f33', 'request_id': 'req-47bb730c-d4c1-47a7-9f51-44d6727b0251'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:08.341452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] The token cache is being invalidate because project eaea892229d5459e94fbcd3dcfe11f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:05:08.341742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:08.342358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-47bb730c-d4c1-47a7-9f51-44d6727b0251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:08.343826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1191/2394] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:05:08 2022] DELETE /identity/v3/projects/eaea892229d5459e94fbcd3dcfe11f33 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:08.349071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c70c56-b398-476f-b562-7ebb587b82c9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.349350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c70c56-b398-476f-b562-7ebb587b82c9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.349616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c70c56-b398-476f-b562-7ebb587b82c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.370105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-99c70c56-b398-476f-b562-7ebb587b82c9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:08.371519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-99c70c56-b398-476f-b562-7ebb587b82c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:08.390083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:08.433294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1204/2395] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:05:08.441377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf0c49-731d-4866-85a3-404161342af3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.441688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf0c49-731d-4866-85a3-404161342af3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.441955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf0c49-731d-4866-85a3-404161342af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.470253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-88bf0c49-731d-4866-85a3-404161342af3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:08.471942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-88bf0c49-731d-4866-85a3-404161342af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:08.483565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1192/2396] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:08.488013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c70c56-b398-476f-b562-7ebb587b82c9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.498518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c70c56-b398-476f-b562-7ebb587b82c9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.504334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.504988 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.505270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.505546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.506206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:05:08.506511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.508133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:08.520337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1205/2397] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:05:08.524739 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf0c49-731d-4866-85a3-404161342af3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.534270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf0c49-731d-4866-85a3-404161342af3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.539916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.540554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.540943 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.541217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.541891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.543804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:08.573944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1193/2398] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:05:08 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:05:08.578247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.588309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505f371-8d9e-4bdd-9d0d-fd3d1a66fa83 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.648308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.648994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.649190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.649442 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.650063 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.651985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:08.675668 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1206/2399] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:05:08 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:05:08.680241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.691564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024b36f-f73f-47fa-976c-33a903637e7a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.697409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.698054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.698327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.698596 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.699241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.700904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:08.710934 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1194/2400] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:05:08.715228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.725548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c657f7d-7cf1-4e46-ac60-457f8b37930b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.730883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e8e64-6b76-492f-b5bb-4942db6914d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.731493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e8e64-6b76-492f-b5bb-4942db6914d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.731768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e8e64-6b76-492f-b5bb-4942db6914d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.732027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e8e64-6b76-492f-b5bb-4942db6914d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.732697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e8e64-6b76-492f-b5bb-4942db6914d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.734432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e8e64-6b76-492f-b5bb-4942db6914d4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:08.745646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1207/2401] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:05:08.749954 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.761131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737e561-ab42-4f44-a964-a087ad522893 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.767096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.767741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.768043 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.768308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] PATH_INFO: `/v3/projects/5d36a4845e374f7598a71d6e525ba111/users/94f67e6c944b469ab9676059c910293b/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.784064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d36a4845e374f7598a71d6e525ba111, user_id=94f67e6c944b469ab9676059c910293b, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.786795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:08.801959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1195/2402] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:05:08 2022] PUT /identity/v3/projects/5d36a4845e374f7598a71d6e525ba111/users/94f67e6c944b469ab9676059c910293b/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:08.812607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63443a7-7375-4875-88d6-2f1092686495 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.812891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63443a7-7375-4875-88d6-2f1092686495 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.813158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63443a7-7375-4875-88d6-2f1092686495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.842348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a63443a7-7375-4875-88d6-2f1092686495 None None] MFA Rules not processed for user `94f67e6c944b469ab9676059c910293b`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:08.843638 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a63443a7-7375-4875-88d6-2f1092686495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:08.909897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1208/2403] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:08.921677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:08.923642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:08.946852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:08.983766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cce02c-242e-4c27-ae5a-f7c8fe364a84 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:08.988166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee9735-c94e-45ae-938a-420d711e87ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:08.988759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee9735-c94e-45ae-938a-420d711e87ea service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:08.989034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee9735-c94e-45ae-938a-420d711e87ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:08.989301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee9735-c94e-45ae-938a-420d711e87ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:08.998354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee9735-c94e-45ae-938a-420d711e87ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:08.999805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee9735-c94e-45ae-938a-420d711e87ea service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:09.009506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1196/2404] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:05:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:05:09.035500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63443a7-7375-4875-88d6-2f1092686495 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:09.056017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:09.102971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63443a7-7375-4875-88d6-2f1092686495 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:09.108210 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:09.108875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.109153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.109426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] PATH_INFO: `/v3/users/94f67e6c944b469ab9676059c910293b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.110619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f67e6c944b469ab9676059c910293b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:09.112601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:09.170252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f67e6c944b469ab9676059c910293b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.178544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f67e6c944b469ab9676059c910293b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.179270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f67e6c944b469ab9676059c910293b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.186422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f67e6c944b469ab9676059c910293b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.200281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f67e6c944b469ab9676059c910293b'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.207289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1209/2405] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:05:09 2022] DELETE /identity/v3/users/94f67e6c944b469ab9676059c910293b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:09.212287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.212607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.212882 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.232957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:09.234194 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:09.297292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1197/2406] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:09.307264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:09.308785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:09.367611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913045c-e063-45f5-9090-a03e9ec2bfcd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:09.371630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafd274-d5bb-40ea-bf1a-b2266bfd19c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:09.372233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafd274-d5bb-40ea-bf1a-b2266bfd19c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.372582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafd274-d5bb-40ea-bf1a-b2266bfd19c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.372858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafd274-d5bb-40ea-bf1a-b2266bfd19c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.382748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafd274-d5bb-40ea-bf1a-b2266bfd19c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:09.384180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafd274-d5bb-40ea-bf1a-b2266bfd19c2 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:09.393655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1210/2407] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:05:09.687322 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:09.689091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:09.703225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9d7f6-1d94-4b6d-bf90-dbd9cf60cebe None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:09.708076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:09.708995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.709347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.709702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] PATH_INFO: `/v3/projects/5d36a4845e374f7598a71d6e525ba111` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.710932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d36a4845e374f7598a71d6e525ba111)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:09.712994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:09.765765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d36a4845e374f7598a71d6e525ba111', 'request_id': 'req-2b40bc91-b595-4441-847b-170abb90e879'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.774710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d36a4845e374f7598a71d6e525ba111', 'request_id': 'req-2b40bc91-b595-4441-847b-170abb90e879'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.783001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d36a4845e374f7598a71d6e525ba111', 'request_id': 'req-2b40bc91-b595-4441-847b-170abb90e879'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.795813 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] The token cache is being invalidate because project 5d36a4845e374f7598a71d6e525ba111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:05:09.796124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.796939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b40bc91-b595-4441-847b-170abb90e879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:05:09.798466 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1198/2408] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:05:09 2022] DELETE /identity/v3/projects/5d36a4845e374f7598a71d6e525ba111 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:09.803422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab877b-3d6e-460a-9fba-0cd9dc3d83f8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.803914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab877b-3d6e-460a-9fba-0cd9dc3d83f8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.804006 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab877b-3d6e-460a-9fba-0cd9dc3d83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.823392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-04ab877b-3d6e-460a-9fba-0cd9dc3d83f8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:09.824604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-04ab877b-3d6e-460a-9fba-0cd9dc3d83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:09.857794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:09.884940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1211/2409] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:05:09.896945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db8820-b2af-4314-826f-288ecc328cc0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.897232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db8820-b2af-4314-826f-288ecc328cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.897505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db8820-b2af-4314-826f-288ecc328cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.919545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-71db8820-b2af-4314-826f-288ecc328cc0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:09.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-71db8820-b2af-4314-826f-288ecc328cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:09.929844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1199/2410] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:09.934161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab877b-3d6e-460a-9fba-0cd9dc3d83f8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:09.944190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab877b-3d6e-460a-9fba-0cd9dc3d83f8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:09.949539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:09.950141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.950423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.950731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.951370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:05:09.951671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:09.953315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:09.965062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1212/2411] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:05:09.969368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db8820-b2af-4314-826f-288ecc328cc0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:09.978630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db8820-b2af-4314-826f-288ecc328cc0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:09.983951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:09.984574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:09.984853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:09.985125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:09.985768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:09.987707 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:10.016857 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1200/2412] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:05:09 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:05:10.020999 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:10.031248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159888-f95b-4d23-81fa-8369a6e0857e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:10.094484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:10.095036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:10.095313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:10.095575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:10.096216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:10.098215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:10.126314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1213/2413] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:05:10 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:05:10.130951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:10.141730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241619d-d056-45a7-b063-36f62cdc39ac admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:10.147246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:10.147853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:10.148143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:10.148406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:10.149084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:10.150717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:10.161164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1201/2414] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:05:10.165241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:10.175604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae51192-eb44-4ce6-b3eb-1039cfe1fa6f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:10.180569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9b97f-2565-4476-bb9c-978aebc075ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:10.181160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b97f-2565-4476-bb9c-978aebc075ae admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:10.181438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b97f-2565-4476-bb9c-978aebc075ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:10.181706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b97f-2565-4476-bb9c-978aebc075ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:10.182345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9b97f-2565-4476-bb9c-978aebc075ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:10.183938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9b97f-2565-4476-bb9c-978aebc075ae admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:10.194658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1214/2415] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:05:10.199068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:10.209932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84be4b-eef5-4805-8301-c1b138ba3bb5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:10.214824 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:10.215479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:10.215760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:10.216038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] PATH_INFO: `/v3/projects/2f6832936c824066a4370cc45c5b0282/users/7880064ff0aa448d9c367c89eba145ff/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:10.218290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:10.233392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6832936c824066a4370cc45c5b0282, user_id=7880064ff0aa448d9c367c89eba145ff, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:10.236088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:10.250937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1202/2416] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:05:10 2022] PUT /identity/v3/projects/2f6832936c824066a4370cc45c5b0282/users/7880064ff0aa448d9c367c89eba145ff/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:10.261901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4149da-10ec-48db-ae97-afff7152b954 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:10.262195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4149da-10ec-48db-ae97-afff7152b954 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:10.262459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4149da-10ec-48db-ae97-afff7152b954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:10.282940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ca4149da-10ec-48db-ae97-afff7152b954 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:10.284225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ca4149da-10ec-48db-ae97-afff7152b954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:10.346768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1215/2417] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:10.356802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:10.358874 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:10.420317 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e600-222f-4aa8-8b33-21436c1eb5db admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:10.424656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:10.425263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:10.425540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:10.425806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:10.435990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:10.437475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:10.447314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1203/2418] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:05:14.448258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13e835-8ddc-4f6c-a601-b6e421ddd6af None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:14.448830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13e835-8ddc-4f6c-a601-b6e421ddd6af None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:14.448929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13e835-8ddc-4f6c-a601-b6e421ddd6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:14.477727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9f13e835-8ddc-4f6c-a601-b6e421ddd6af None None] MFA Rules not processed for user `7880064ff0aa448d9c367c89eba145ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:14.478870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9f13e835-8ddc-4f6c-a601-b6e421ddd6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:14.529002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:14.557809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1216/2419] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:14.562400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:14.572661 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0bbc7-9079-4753-b190-304041df80bc service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:14.584569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:14.585375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:14.585722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:14.586081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:14.586807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:14.589003 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:14.620538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1204/2420] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:05:14 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:05:14.623933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13e835-8ddc-4f6c-a601-b6e421ddd6af None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:14.635944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13e835-8ddc-4f6c-a601-b6e421ddd6af None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:14.713631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:14.714304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:14.714602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:14.714872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:14.715514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:14.717517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:14.745268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1217/2421] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:05:14 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:05:14.749484 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:14.760883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd37bc-cab2-4a9b-af24-4c20e2e0a30c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:14.765774 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:14.766371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:14.766698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:14.766970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:14.767597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:14.769257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:14.781873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1205/2422] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:05:14.786284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:14.797607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9d189-0ac8-41f6-b0b1-4bd733fb04d3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:14.802512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:14.803188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:14.803466 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:14.803732 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] PATH_INFO: `/v3/projects/21add0dce4a24d13816aa9987bc7af81/users/2f05415eeaf24ab5aaafdeb100f48890/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:14.819530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21add0dce4a24d13816aa9987bc7af81, user_id=2f05415eeaf24ab5aaafdeb100f48890, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:14.822230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:14.839784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1218/2423] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:05:14 2022] PUT /identity/v3/projects/21add0dce4a24d13816aa9987bc7af81/users/2f05415eeaf24ab5aaafdeb100f48890/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:14.841054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:14.852666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5dc18-22f2-4afb-8400-099f45909953 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:14.912582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef768bd-8cd8-4462-8d08-2cf3aca99465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:14.913243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef768bd-8cd8-4462-8d08-2cf3aca99465 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:14.913472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef768bd-8cd8-4462-8d08-2cf3aca99465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:14.913743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef768bd-8cd8-4462-8d08-2cf3aca99465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:14.914401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef768bd-8cd8-4462-8d08-2cf3aca99465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:14.916134 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef768bd-8cd8-4462-8d08-2cf3aca99465 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:14.927260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1206/2424] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:05:14.931411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:14.942545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb65a5-bf90-4707-ac15-815f0315e207 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:14.947166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:14.947841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:14.948244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:14.948461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] PATH_INFO: `/v3/projects/21add0dce4a24d13816aa9987bc7af81/users/2f05415eeaf24ab5aaafdeb100f48890/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:14.950364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21add0dce4a24d13816aa9987bc7af81, user_id=2f05415eeaf24ab5aaafdeb100f48890, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:14.957501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:14.971743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1219/2425] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:05:14 2022] PUT /identity/v3/projects/21add0dce4a24d13816aa9987bc7af81/users/2f05415eeaf24ab5aaafdeb100f48890/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:05:19.377607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bad2e-f7e7-479a-859b-79af7619de43 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:19.378504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bad2e-f7e7-479a-859b-79af7619de43 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:19.378939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bad2e-f7e7-479a-859b-79af7619de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:19.426356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fa8bad2e-f7e7-479a-859b-79af7619de43 None None] MFA Rules not processed for user `2f05415eeaf24ab5aaafdeb100f48890`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:19.427947 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fa8bad2e-f7e7-479a-859b-79af7619de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:19.515376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:19.559793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1207/2426] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:19.573958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:19.575856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:19.662761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602503c9-fdcb-4c76-bae0-54ea6bf5813b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:19.667718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:19.668535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:19.668967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:19.669333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:19.765592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:19.796773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:19.803694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:19.815690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1220/2427] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:05:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:05:19.988401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c462-9e0a-4f20-807b-13e8414457dc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:19.989069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c462-9e0a-4f20-807b-13e8414457dc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:19.989411 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c462-9e0a-4f20-807b-13e8414457dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:20.012626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3a76c462-9e0a-4f20-807b-13e8414457dc None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:20.014306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3a76c462-9e0a-4f20-807b-13e8414457dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:20.152821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1208/2428] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:05:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:20.169677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:20.171557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:20.313694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432c331-f1ff-46ef-b725-034e069d44f7 service glance] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:20.325568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444628-12cb-41ef-88dc-822c171e5a2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:20.326964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444628-12cb-41ef-88dc-822c171e5a2c service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:20.327344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444628-12cb-41ef-88dc-822c171e5a2c service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:20.327710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444628-12cb-41ef-88dc-822c171e5a2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:20.344894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444628-12cb-41ef-88dc-822c171e5a2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:20.346777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49444628-12cb-41ef-88dc-822c171e5a2c service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:20.358188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1221/2429] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:05:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:05:29.676683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7146f9-c9e9-4d0e-94cc-f3603e6bd2af None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:29.677179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7146f9-c9e9-4d0e-94cc-f3603e6bd2af None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:29.677295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7146f9-c9e9-4d0e-94cc-f3603e6bd2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:29.699291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-be7146f9-c9e9-4d0e-94cc-f3603e6bd2af None None] MFA Rules not processed for user `7880064ff0aa448d9c367c89eba145ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:29.700436 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-be7146f9-c9e9-4d0e-94cc-f3603e6bd2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:29.708201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1209/2430] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:30.353838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=40 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:30.869296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0dd0-a01f-4925-8282-cc342fa58b30 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:30.869585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0dd0-a01f-4925-8282-cc342fa58b30 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:30.869873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a0dd0-a01f-4925-8282-cc342fa58b30 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:30.871432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1222/2431] 10.222.0.43 () {54 vars in 961 bytes} [Tue Sep 13 18:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:05:30.876500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8047-b27a-4229-9b57-eed3d5e8daaf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:30.877304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8047-b27a-4229-9b57-eed3d5e8daaf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:30.877509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd8047-b27a-4229-9b57-eed3d5e8daaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:30.896809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-05dd8047-b27a-4229-9b57-eed3d5e8daaf None None] MFA Rules not processed for user `c6ce465c1ee84703b7b44d33a91d7017`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:30.898090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-05dd8047-b27a-4229-9b57-eed3d5e8daaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:30.963094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1210/2432] 10.222.0.43 () {58 vars in 1058 bytes} [Tue Sep 13 18:05:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:05:30.971423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0dd0-a01f-4925-8282-cc342fa58b30 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:30.987786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a0dd0-a01f-4925-8282-cc342fa58b30 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:30.996162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:30.996875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:30.997225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:30.997565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:31.000310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:31.013543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:31.015342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:31.027105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1223/2433] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:05:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:05:33.949365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83511f35-9ed6-4463-aec0-7d4e412b97d4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:33.949935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83511f35-9ed6-4463-aec0-7d4e412b97d4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:33.949935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83511f35-9ed6-4463-aec0-7d4e412b97d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:33.970509 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-83511f35-9ed6-4463-aec0-7d4e412b97d4 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:33.971993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-83511f35-9ed6-4463-aec0-7d4e412b97d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:33.982124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1211/2434] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:05:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:34.011120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:05:34.024302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39e4a0-e4a2-412b-96ff-40284473447f service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:05:34.031980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f85541-3402-419e-bc53-deb12635a3be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:05:34.032874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f85541-3402-419e-bc53-deb12635a3be service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:34.033341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f85541-3402-419e-bc53-deb12635a3be service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:34.033797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f85541-3402-419e-bc53-deb12635a3be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:34.057428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f85541-3402-419e-bc53-deb12635a3be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:05:34.061021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f85541-3402-419e-bc53-deb12635a3be service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:05:34.076464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1224/2435] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:05:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:05:43.964528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=72 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:44.069214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=12 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:45.445873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:45.446482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:45.446482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:45.465408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] MFA Rules not processed for user `60800cf2774e409d9c1b6f4992669991`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:45.466758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:45.526748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1212/2436] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:46.626924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051d2c4-7503-4a60-90d0-b38355eab5d4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:05:46.627790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051d2c4-7503-4a60-90d0-b38355eab5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:05:46.627905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051d2c4-7503-4a60-90d0-b38355eab5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:05:46.647688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0051d2c4-7503-4a60-90d0-b38355eab5d4 None None] MFA Rules not processed for user `e68ecd9b3ca643539b109f75bb242d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:05:46.649206 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0051d2c4-7503-4a60-90d0-b38355eab5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:05:46.704865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1225/2437] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:05:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:05:55.518871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=25 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:05:56.697921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=25 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:06:03.300300 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:03.301910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:03.361622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a99de3-727c-4d0d-9277-371ead5555a3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:03.365221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:03.365832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:03.366111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:03.366389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:03.375912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:03.377791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:03.387677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1213/2438] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:06:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:06:13.383610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=26 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:06:20.864202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592511d-11db-4db0-befb-7041f497e0f5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:20.864776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592511d-11db-4db0-befb-7041f497e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:20.864776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592511d-11db-4db0-befb-7041f497e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:20.885229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e592511d-11db-4db0-befb-7041f497e0f5 None None] MFA Rules not processed for user `0625b3113ce044bbac4f6142eaade11c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:06:20.886702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e592511d-11db-4db0-befb-7041f497e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:20.942534 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1226/2439] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:06:30.935955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=25 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:06:46.839613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:46.841361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:46.923932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb13b35-2338-41d2-9f9d-b81c3938b214 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:46.930284 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:46.930886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:46.931161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:46.931429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:46.933394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:46.944094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:46.945743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:46.955616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1214/2440] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:06:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:06:47.858084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592511d-11db-4db0-befb-7041f497e0f5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:47.868748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592511d-11db-4db0-befb-7041f497e0f5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:47.873324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:47.874017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:47.874378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:47.874721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:47.876645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:47.891411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:47.892907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:47.904359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1227/2441] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:06:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:06:48.642802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:48.644463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:48.661451 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af143da5-d512-4e1f-8829-71f6b2a29315 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:48.668871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:48.669670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:48.670011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:48.670340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:48.672372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:48.684644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:48.686506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:48.697540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1215/2442] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:06:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:06:50.580131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:50.589213 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686e9df-a129-487f-9938-4b5256cc3026 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:50.593723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:50.594294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:50.594574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:50.594846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:50.596797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:50.663285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:50.664820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:50.674758 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1228/2443] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:06:51.907467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:51.918447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06fc2c-a6f7-47c9-951a-59aadd943f09 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:51.922963 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:51.923567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:51.923843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:51.924114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:51.925937 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:51.939770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:51.941249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:51.952587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1216/2444] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:06:54.705119 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:54.706669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:54.719459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6841c8-0543-4139-894c-0c7fa436bda3 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:54.725204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:54.725839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:54.726400 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:54.732590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] PATH_INFO: `/v3/users/60800cf2774e409d9c1b6f4992669991` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:54.732590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60800cf2774e409d9c1b6f4992669991)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:54.734978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:54.784395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60800cf2774e409d9c1b6f4992669991'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:54.792218 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60800cf2774e409d9c1b6f4992669991'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:54.792886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60800cf2774e409d9c1b6f4992669991'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:54.799869 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60800cf2774e409d9c1b6f4992669991'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:54.811405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3b5ed896-e9f1-4720-82cf-0e78e89d58a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60800cf2774e409d9c1b6f4992669991'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:54.817725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1229/2445] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:06:54 2022] DELETE /identity/v3/users/60800cf2774e409d9c1b6f4992669991 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:06:55.001633 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.019790 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c2c01-8a79-4887-ae1e-2b69f042d19b service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.078239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.078867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.079137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.079425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] PATH_INFO: `/v3/projects/2eae4acecf6c4961830ead78d71b2ad3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.080579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eae4acecf6c4961830ead78d71b2ad3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.082508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.131563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eae4acecf6c4961830ead78d71b2ad3', 'request_id': 'req-c21e05c6-f9f1-41e9-b069-6e186b04daf0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:55.141363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eae4acecf6c4961830ead78d71b2ad3', 'request_id': 'req-c21e05c6-f9f1-41e9-b069-6e186b04daf0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:55.148640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eae4acecf6c4961830ead78d71b2ad3', 'request_id': 'req-c21e05c6-f9f1-41e9-b069-6e186b04daf0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:55.149865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:06:55.165488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] The token cache is being invalidate because project 2eae4acecf6c4961830ead78d71b2ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:06:55.165779 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:55.166423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c21e05c6-f9f1-41e9-b069-6e186b04daf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:06:55.167969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1217/2446] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:06:55 2022] DELETE /identity/v3/projects/2eae4acecf6c4961830ead78d71b2ad3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:06:55.173110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ecbb9-81f5-418c-8f93-c32a63816565 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.173401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ecbb9-81f5-418c-8f93-c32a63816565 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.173680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ecbb9-81f5-418c-8f93-c32a63816565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.195461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ed8ecbb9-81f5-418c-8f93-c32a63816565 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:06:55.196825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ed8ecbb9-81f5-418c-8f93-c32a63816565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:55.244569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:06:55.258504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1230/2447] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:06:55.266740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875342ba-19ad-451d-827f-f51982c714ce None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.267015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875342ba-19ad-451d-827f-f51982c714ce None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.267286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875342ba-19ad-451d-827f-f51982c714ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.287368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-875342ba-19ad-451d-827f-f51982c714ce None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:06:55.288570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-875342ba-19ad-451d-827f-f51982c714ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:55.297085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1218/2448] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:06:55.302838 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ecbb9-81f5-418c-8f93-c32a63816565 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.312531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ecbb9-81f5-418c-8f93-c32a63816565 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.318625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.319256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.319516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.319776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.320444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:06:55.320787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.322379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.333967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1231/2449] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:06:55.338394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875342ba-19ad-451d-827f-f51982c714ce None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.348952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875342ba-19ad-451d-827f-f51982c714ce None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.355032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.355622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.355895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.356176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.356819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.358807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.387944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1219/2450] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:06:55 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:06:55.392286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.402712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7c227-463c-427b-b693-c8e0a130dfbc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.463964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.464518 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.464793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.465079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.465781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.467771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.491640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1232/2451] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:06:55 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:06:55.495993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.507329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7579e-8c5a-47c3-b906-92b6cf8e4a11 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.512801 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.513405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.513683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.513953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.514601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.516230 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.526660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1220/2452] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:06:55.530735 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.543936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4225ba4-394c-44c6-a773-962b450f65ea admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.552142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7356e-8bbe-4888-91ea-64bef6cb6182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.553086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7356e-8bbe-4888-91ea-64bef6cb6182 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.553485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7356e-8bbe-4888-91ea-64bef6cb6182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.553864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7356e-8bbe-4888-91ea-64bef6cb6182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.554903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7356e-8bbe-4888-91ea-64bef6cb6182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.556707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7356e-8bbe-4888-91ea-64bef6cb6182 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.567712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1233/2453] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:06:55.571873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.582364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16f3a6-c3d9-4200-9ee0-f591721808a5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.587625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.588262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.588578 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.588845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] PATH_INFO: `/v3/projects/2232724e013f421587e66d0c868f56bc/users/3a68ef1a47ef49e7b88bde653c0687aa/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.603901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2232724e013f421587e66d0c868f56bc, user_id=3a68ef1a47ef49e7b88bde653c0687aa, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.606592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.622453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1221/2454] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:06:55 2022] PUT /identity/v3/projects/2232724e013f421587e66d0c868f56bc/users/3a68ef1a47ef49e7b88bde653c0687aa/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:06:55.633405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.633693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.633969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.664901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] MFA Rules not processed for user `3a68ef1a47ef49e7b88bde653c0687aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:06:55.666182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:55.735781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1234/2455] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:06:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:06:55.746820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:06:55.748688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:06:55.810151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3439107-7bf8-4961-b1e0-d8845944700c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:06:55.814064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:06:55.814670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:06:55.814948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:06:55.815216 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:06:55.824073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:06:55.825558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:06:55.834641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1222/2456] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:06:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:02.256264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:02.257787 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:02.314183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:02.337896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb72fb-c901-4b8e-92c4-41e899143734 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:02.343795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:02.344457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:02.344747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:02.345013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:02.346947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:02.359694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:02.361488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:02.373094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1235/2457] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:03.893309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:03.895971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:03.957366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fcae6-2f57-4606-bc0c-72125c3605a2 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:03.961246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ce56c-aef8-494d-88bf-ec3d04edd78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:03.961923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ce56c-aef8-494d-88bf-ec3d04edd78b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:03.962209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ce56c-aef8-494d-88bf-ec3d04edd78b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:03.962472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ce56c-aef8-494d-88bf-ec3d04edd78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:03.964418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-039ce56c-aef8-494d-88bf-ec3d04edd78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:03.974311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ce56c-aef8-494d-88bf-ec3d04edd78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:03.975752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ce56c-aef8-494d-88bf-ec3d04edd78b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:03.983864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1223/2458] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:04.374141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:04.438040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba9426-fd49-4bb9-8fb8-907ad88efc5a service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:04.443034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:04.443678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:04.443962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:04.444217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] PATH_INFO: `/v3/users/3a68ef1a47ef49e7b88bde653c0687aa` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:04.447587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a68ef1a47ef49e7b88bde653c0687aa)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:04.449559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:04.502531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a68ef1a47ef49e7b88bde653c0687aa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:04.511379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a68ef1a47ef49e7b88bde653c0687aa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:04.512019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a68ef1a47ef49e7b88bde653c0687aa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:04.519753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a68ef1a47ef49e7b88bde653c0687aa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:04.533517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a68ef1a47ef49e7b88bde653c0687aa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:04.540087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1236/2459] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:04 2022] DELETE /identity/v3/users/3a68ef1a47ef49e7b88bde653c0687aa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:04.545180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:04.545377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:04.545710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:04.565522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:04.566765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:04.594433 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:04.629612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1224/2460] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:04.639977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:04.641659 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:04.700208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbda41-162e-4eb9-be01-780f435f8563 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:04.704253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dfdb8-c230-4f6b-9b76-8526dfe78c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:04.704889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfdb8-c230-4f6b-9b76-8526dfe78c01 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:04.705162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfdb8-c230-4f6b-9b76-8526dfe78c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:04.705430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfdb8-c230-4f6b-9b76-8526dfe78c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:04.714578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dfdb8-c230-4f6b-9b76-8526dfe78c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:04.715982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dfdb8-c230-4f6b-9b76-8526dfe78c01 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:04.720834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:04.726542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1237/2461] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:05.070462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.072023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:05.084019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3bc61-5032-4426-9b91-aa8c03105f0e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.089534 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.090142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.090574 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.091033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] PATH_INFO: `/v3/projects/2232724e013f421587e66d0c868f56bc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.092179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2232724e013f421587e66d0c868f56bc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.094199 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.142733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2232724e013f421587e66d0c868f56bc', 'request_id': 'req-1845397b-c1c5-4496-adec-a67eb97d4ed9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:05.152076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2232724e013f421587e66d0c868f56bc', 'request_id': 'req-1845397b-c1c5-4496-adec-a67eb97d4ed9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:05.158704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2232724e013f421587e66d0c868f56bc', 'request_id': 'req-1845397b-c1c5-4496-adec-a67eb97d4ed9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:05.169350 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] The token cache is being invalidate because project 2232724e013f421587e66d0c868f56bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:05.169619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:05.170283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1845397b-c1c5-4496-adec-a67eb97d4ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:05.171811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1225/2462] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:05 2022] DELETE /identity/v3/projects/2232724e013f421587e66d0c868f56bc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:05.176871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04a63-1e64-4f91-b084-a074a90886b7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.177157 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04a63-1e64-4f91-b084-a074a90886b7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.177431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de04a63-1e64-4f91-b084-a074a90886b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.197135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2de04a63-1e64-4f91-b084-a074a90886b7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:05.198322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2de04a63-1e64-4f91-b084-a074a90886b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:05.261015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1238/2463] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:05.269469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0105-4dcb-43c7-93eb-dde3fc22f837 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.269864 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0105-4dcb-43c7-93eb-dde3fc22f837 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.270204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc0105-4dcb-43c7-93eb-dde3fc22f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.290668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d0fc0105-4dcb-43c7-93eb-dde3fc22f837 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:05.291849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d0fc0105-4dcb-43c7-93eb-dde3fc22f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:05.299785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1226/2464] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:05.304053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de04a63-1e64-4f91-b084-a074a90886b7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.314131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de04a63-1e64-4f91-b084-a074a90886b7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.319625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.320316 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.320859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.321133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.321793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:07:05.322099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.323709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.336372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1239/2465] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 18:07:05.341309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc0105-4dcb-43c7-93eb-dde3fc22f837 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.350831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc0105-4dcb-43c7-93eb-dde3fc22f837 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.355643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.356243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.356563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.356827 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.357465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.359361 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.387137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1227/2466] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:05 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:05.391382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.402473 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a3547-4198-4a41-9548-3bdc4383a531 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.464973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.465620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.465897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.466163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.466800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.468819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.492818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1240/2467] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:05 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:05.497254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.510073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d3beb-2ee7-4d07-ad5d-60c6d6e63936 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.516313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.516957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.517236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.517508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.518157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.519780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.531435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1228/2468] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:05.535733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.546061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3731354-2ee0-4cec-9528-e6d9750861fe admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.551254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ce5a2-a344-4184-9f63-2aba0ddb611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.551841 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce5a2-a344-4184-9f63-2aba0ddb611f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.552093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce5a2-a344-4184-9f63-2aba0ddb611f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.552405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce5a2-a344-4184-9f63-2aba0ddb611f admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.553087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ce5a2-a344-4184-9f63-2aba0ddb611f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.554723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ce5a2-a344-4184-9f63-2aba0ddb611f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.565591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1241/2469] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:05.570008 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.580398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cceed4-1844-4a53-a3fc-4b8b60363ed7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.590863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.591557 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.591826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.592098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] PATH_INFO: `/v3/projects/4092dfe931a8477c9e2d944f21833a3a/users/1751990e757f42c8be042be33f318678/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.607343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4092dfe931a8477c9e2d944f21833a3a, user_id=1751990e757f42c8be042be33f318678, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.610034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.623932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1229/2470] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:05 2022] PUT /identity/v3/projects/4092dfe931a8477c9e2d944f21833a3a/users/1751990e757f42c8be042be33f318678/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:05.635254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd920-62ad-48fa-a21d-959f68f74517 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.635731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd920-62ad-48fa-a21d-959f68f74517 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.636069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd920-62ad-48fa-a21d-959f68f74517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.665614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-677cd920-62ad-48fa-a21d-959f68f74517 None None] MFA Rules not processed for user `1751990e757f42c8be042be33f318678`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:05.666853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-677cd920-62ad-48fa-a21d-959f68f74517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:05.722021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:05.736977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1242/2471] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:05.738196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.791717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:05.808749 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06f534-746d-4f8b-84d9-ba2c6c26c1f9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.816883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.817796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.818378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.818804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.819831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.823265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.857319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1230/2472] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:05 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:05.863242 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cd920-62ad-48fa-a21d-959f68f74517 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.874817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cd920-62ad-48fa-a21d-959f68f74517 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.932727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.933343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.933623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.933891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.934530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.936470 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.960238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1243/2473] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:05 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:05.964639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:05.975012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedde3d-89fa-4352-9b5e-5ad74f09f85d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:05.979997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:05.980636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:05.980902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:05.981169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:05.981849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:05.983467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:05.994425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1231/2474] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:05.999376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:06.015428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bf268-38b7-401d-85c9-193b94e5dd3c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:06.023183 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:06.024228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.024546 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.024815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] PATH_INFO: `/v3/projects/05ff1d7c90434538944e39c4604b5463/users/1583b24e384746c5b67ea2ef5df92693/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.040914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ff1d7c90434538944e39c4604b5463, user_id=1583b24e384746c5b67ea2ef5df92693, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:06.043567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:06.058596 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1244/2475] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:05 2022] PUT /identity/v3/projects/05ff1d7c90434538944e39c4604b5463/users/1583b24e384746c5b67ea2ef5df92693/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:06.063265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:06.079030 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5429da8-cc22-4766-a658-e0a82952364f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:06.146343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61898d3f-9c54-455f-92b5-6df926121840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:06.146960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898d3f-9c54-455f-92b5-6df926121840 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.147241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898d3f-9c54-455f-92b5-6df926121840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.147513 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61898d3f-9c54-455f-92b5-6df926121840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.148170 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61898d3f-9c54-455f-92b5-6df926121840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:06.149931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61898d3f-9c54-455f-92b5-6df926121840 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:06.160906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1232/2476] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:06.165604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:06.176111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff27fba-12f9-4021-91db-acbaac4eede0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:06.181036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:06.181708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.182019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.182298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] PATH_INFO: `/v3/projects/05ff1d7c90434538944e39c4604b5463/users/1583b24e384746c5b67ea2ef5df92693/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.184215 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ff1d7c90434538944e39c4604b5463, user_id=1583b24e384746c5b67ea2ef5df92693, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:06.186896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:06.198118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1245/2477] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:06 2022] PUT /identity/v3/projects/05ff1d7c90434538944e39c4604b5463/users/1583b24e384746c5b67ea2ef5df92693/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:06.209878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80f40-c3e5-47ee-83fe-10ed50af59dc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.210163 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80f40-c3e5-47ee-83fe-10ed50af59dc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.210461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80f40-c3e5-47ee-83fe-10ed50af59dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.239607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-12e80f40-c3e5-47ee-83fe-10ed50af59dc None None] MFA Rules not processed for user `1583b24e384746c5b67ea2ef5df92693`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:06.240840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-12e80f40-c3e5-47ee-83fe-10ed50af59dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:06.310381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1233/2478] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:06.323941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:06.326150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:06.393178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d03cc-2c99-42e1-b5d3-ce5eecf53501 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:06.397002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05c92c-3fc8-4576-b46a-961e0a998bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:06.397623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05c92c-3fc8-4576-b46a-961e0a998bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.397902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05c92c-3fc8-4576-b46a-961e0a998bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.398166 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05c92c-3fc8-4576-b46a-961e0a998bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.407601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05c92c-3fc8-4576-b46a-961e0a998bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:06.409067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05c92c-3fc8-4576-b46a-961e0a998bd1 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:06.418858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1246/2479] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:06.495481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80f40-c3e5-47ee-83fe-10ed50af59dc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:06.506328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80f40-c3e5-47ee-83fe-10ed50af59dc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:06.510794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:06.511391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.511667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.511936 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.559218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:06.573692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:06.575112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:06.584865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1234/2480] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:06.717984 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.719467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.719899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.747484 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:06.748795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:06.795239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:06.807967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1247/2481] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:07:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:06.848976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:06.850517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:06.921914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33fe9d-2076-4805-93b7-675265c236c9 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:06.927644 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:06.928247 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:06.928554 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:06.928823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:06.937232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:06.938846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:06.947034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1235/2482] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:07:10.519833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:10.521230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:10.613840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14607d21-4a7a-4a06-8d7b-280bdd81a46a None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:10.621093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e88905-52b6-4d92-b1e2-e1328f06e039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:10.621809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e88905-52b6-4d92-b1e2-e1328f06e039 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:10.622082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e88905-52b6-4d92-b1e2-e1328f06e039 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:10.622361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e88905-52b6-4d92-b1e2-e1328f06e039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:10.625146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-59e88905-52b6-4d92-b1e2-e1328f06e039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:10.696091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e88905-52b6-4d92-b1e2-e1328f06e039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:10.698769 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e88905-52b6-4d92-b1e2-e1328f06e039 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:10.710225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1248/2483] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:15.981368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:16.064972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30601f-d783-417e-bcdf-ae245cac3c88 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:16.071263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:16.074046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:16.074046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:16.075143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] PATH_INFO: `/v3/users/1751990e757f42c8be042be33f318678` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:16.076409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1751990e757f42c8be042be33f318678)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:16.078572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:16.148791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1751990e757f42c8be042be33f318678'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.157158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1751990e757f42c8be042be33f318678'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.157785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1751990e757f42c8be042be33f318678'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.168173 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1751990e757f42c8be042be33f318678'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.181104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:16.183666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1751990e757f42c8be042be33f318678'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.194127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1236/2484] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:15 2022] DELETE /identity/v3/users/1751990e757f42c8be042be33f318678 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:16.196491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:16.197022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:16.197022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:16.217270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:16.218759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:16.285822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1249/2485] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:16.296698 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:16.298200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:16.360946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930b984-99c0-458f-a553-fdeaf060aecd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:16.364959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:16.365746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:16.366028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:16.366392 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:16.385780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:16.387523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:16.398771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1237/2486] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:16.689253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:16.690773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:16.703554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9f29c-93a9-4e43-a39c-896ae5c8f0e7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:16.710246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:16.710893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:16.711172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:16.711576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] PATH_INFO: `/v3/projects/4092dfe931a8477c9e2d944f21833a3a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:16.712759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4092dfe931a8477c9e2d944f21833a3a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:16.714674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:16.751303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:16.769567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4092dfe931a8477c9e2d944f21833a3a', 'request_id': 'req-b110dafc-35ed-4a7a-8d71-457b21b6fe98'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.777268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4092dfe931a8477c9e2d944f21833a3a', 'request_id': 'req-b110dafc-35ed-4a7a-8d71-457b21b6fe98'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.783211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4092dfe931a8477c9e2d944f21833a3a', 'request_id': 'req-b110dafc-35ed-4a7a-8d71-457b21b6fe98'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.794784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] The token cache is being invalidate because project 4092dfe931a8477c9e2d944f21833a3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:16.795076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.795721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.797267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1250/2487] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:16 2022] DELETE /identity/v3/projects/4092dfe931a8477c9e2d944f21833a3a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:16.801320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:16.810753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553be4f-870e-48de-a0a9-c36715d7f416 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:16.866132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:16.866684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:16.866957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:16.867231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] PATH_INFO: `/v3/users/1583b24e384746c5b67ea2ef5df92693` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:16.868472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1583b24e384746c5b67ea2ef5df92693)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:16.870850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:16.921911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1583b24e384746c5b67ea2ef5df92693'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.931453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1583b24e384746c5b67ea2ef5df92693'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.932319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1583b24e384746c5b67ea2ef5df92693'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.940035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1583b24e384746c5b67ea2ef5df92693'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.952245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87e755c0-81b0-4396-8f39-256991ddf49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1583b24e384746c5b67ea2ef5df92693'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:16.958329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1238/2488] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:16 2022] DELETE /identity/v3/users/1583b24e384746c5b67ea2ef5df92693 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:17.245025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:17.246624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:17.260404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110dafc-35ed-4a7a-8d71-457b21b6fe98 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:17.317238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:17.317874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.318153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.318426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] PATH_INFO: `/v3/projects/05ff1d7c90434538944e39c4604b5463` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.319540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05ff1d7c90434538944e39c4604b5463)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:17.321554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:17.370985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05ff1d7c90434538944e39c4604b5463', 'request_id': 'req-202c3c8f-35c6-4f35-90c0-5750d067206c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:17.380237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05ff1d7c90434538944e39c4604b5463', 'request_id': 'req-202c3c8f-35c6-4f35-90c0-5750d067206c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:17.386010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05ff1d7c90434538944e39c4604b5463', 'request_id': 'req-202c3c8f-35c6-4f35-90c0-5750d067206c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:17.396830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] The token cache is being invalidate because project 05ff1d7c90434538944e39c4604b5463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:17.397135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:17.397799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-202c3c8f-35c6-4f35-90c0-5750d067206c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:17.399430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1251/2489] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:17 2022] DELETE /identity/v3/projects/05ff1d7c90434538944e39c4604b5463 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:17.405302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113181a-5ced-4947-9457-7fa9b2d14880 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.405796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113181a-5ced-4947-9457-7fa9b2d14880 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.405908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113181a-5ced-4947-9457-7fa9b2d14880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.425761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2113181a-5ced-4947-9457-7fa9b2d14880 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:17.426951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2113181a-5ced-4947-9457-7fa9b2d14880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:17.436629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:17.498122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1239/2490] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:17.506929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7d3d-59c2-4e89-9e6c-87d508f1e3b6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.507298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7d3d-59c2-4e89-9e6c-87d508f1e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.507637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7d3d-59c2-4e89-9e6c-87d508f1e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.528382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-135d7d3d-59c2-4e89-9e6c-87d508f1e3b6 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:17.530097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-135d7d3d-59c2-4e89-9e6c-87d508f1e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:17.538872 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1252/2491] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:17.543514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113181a-5ced-4947-9457-7fa9b2d14880 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:17.554624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113181a-5ced-4947-9457-7fa9b2d14880 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:17.560196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:17.561305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.561586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.561859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.562549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:07:17.562854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:17.564589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:17.576812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1240/2492] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:07:17.581481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d7d3d-59c2-4e89-9e6c-87d508f1e3b6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:17.590695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d7d3d-59c2-4e89-9e6c-87d508f1e3b6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:17.595260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:17.595858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.596131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.596394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.597050 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:17.598946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:17.631331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1253/2493] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:17 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:17.632536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:17.642825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5da91b-bc7e-4c45-924e-e47320ef216c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:17.713290 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:17.713970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.714251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.714515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.715149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:17.717079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:17.744586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1241/2494] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:17 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:17.749886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:17.765309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28a386-2d6a-4b6c-b156-d8561a6547e0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:17.770591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:17.771338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.771774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.772165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.773188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:17.774605 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:17.786258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1254/2495] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:17.790901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:17.801823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607ce34-bf81-438c-b47f-cd47af221a3a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:17.807629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7321442-c341-43f6-9038-e12f145e366d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:17.808262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321442-c341-43f6-9038-e12f145e366d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.808604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321442-c341-43f6-9038-e12f145e366d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.808878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7321442-c341-43f6-9038-e12f145e366d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.809536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7321442-c341-43f6-9038-e12f145e366d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:17.811199 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7321442-c341-43f6-9038-e12f145e366d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:17.826768 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1242/2496] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:17.830687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:17.841309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01733f-6efb-4005-8483-22d01277a474 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:17.846216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:17.847076 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.847540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.847991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] PATH_INFO: `/v3/projects/2c2939d1d73e44c6bf86b4adfe781a4c/users/00c14a8bc2014271b2b3478284eee189/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.850946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:17.871737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2939d1d73e44c6bf86b4adfe781a4c, user_id=00c14a8bc2014271b2b3478284eee189, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:17.874623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:17.890731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1255/2497] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:17 2022] PUT /identity/v3/projects/2c2939d1d73e44c6bf86b4adfe781a4c/users/00c14a8bc2014271b2b3478284eee189/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:17.901138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086ce1-b07d-4ade-a116-3f6c067dff7c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:17.901429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086ce1-b07d-4ade-a116-3f6c067dff7c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:17.901704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086ce1-b07d-4ade-a116-3f6c067dff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:17.930314 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-52086ce1-b07d-4ade-a116-3f6c067dff7c None None] MFA Rules not processed for user `00c14a8bc2014271b2b3478284eee189`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:17.931777 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-52086ce1-b07d-4ade-a116-3f6c067dff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:18.022758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1243/2498] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:18.027423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.090150 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a54c-a546-4601-8a46-eb946cb849c9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.094923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.095547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.095824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.096092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.096770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.098664 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.126542 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1256/2499] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:18 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:18.131119 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086ce1-b07d-4ade-a116-3f6c067dff7c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.141800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086ce1-b07d-4ade-a116-3f6c067dff7c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.149660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:18.199301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.199909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.200186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.200480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.201124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.203121 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.226305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1244/2500] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:18 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:18.231046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.242389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e82b82-8c58-427f-bc7e-7a88ff716a34 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.248096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.248728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.249019 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.249290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.249960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.251585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.263302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1257/2501] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:18.267669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.278016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61ca50-4f22-4f4d-acc7-1276abf8507a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.283082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.283705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.283976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.284241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] PATH_INFO: `/v3/projects/a93940372597409688fc94426f6b7670/users/5067f308ecbb4b3bbd8e780c8fa1f621/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.299506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93940372597409688fc94426f6b7670, user_id=5067f308ecbb4b3bbd8e780c8fa1f621, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.302217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.316111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1245/2502] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:18 2022] PUT /identity/v3/projects/a93940372597409688fc94426f6b7670/users/5067f308ecbb4b3bbd8e780c8fa1f621/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:18.320244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.330528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc651d-df7f-4aa6-b4bf-d4686db884bc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.392256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d22e4-16fa-4c68-b107-666aeba37499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.392843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d22e4-16fa-4c68-b107-666aeba37499 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.393121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d22e4-16fa-4c68-b107-666aeba37499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.393393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d22e4-16fa-4c68-b107-666aeba37499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.394044 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d22e4-16fa-4c68-b107-666aeba37499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.395684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d22e4-16fa-4c68-b107-666aeba37499 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.405819 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1258/2503] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:18.410351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.421249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b16c1d-4b84-4d79-b88a-a90c3ef3be7d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.426470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.427083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.427367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.427631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] PATH_INFO: `/v3/projects/a93940372597409688fc94426f6b7670/users/5067f308ecbb4b3bbd8e780c8fa1f621/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.429906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93940372597409688fc94426f6b7670, user_id=5067f308ecbb4b3bbd8e780c8fa1f621, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.432620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.443056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1246/2504] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:18 2022] PUT /identity/v3/projects/a93940372597409688fc94426f6b7670/users/5067f308ecbb4b3bbd8e780c8fa1f621/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:18.455407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe321b9-5156-4ef4-a0af-0fb3e1e1f82e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.455759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe321b9-5156-4ef4-a0af-0fb3e1e1f82e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.456107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe321b9-5156-4ef4-a0af-0fb3e1e1f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.488618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dbe321b9-5156-4ef4-a0af-0fb3e1e1f82e None None] MFA Rules not processed for user `5067f308ecbb4b3bbd8e780c8fa1f621`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:18.489814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dbe321b9-5156-4ef4-a0af-0fb3e1e1f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:18.532958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:18.565413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1259/2505] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:18.578129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.579951 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:18.640259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad4847-1960-4729-bf07-b3867b2c3f20 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.644197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.644757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.645025 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.645287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.657412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.658899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.668812 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1247/2506] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:18.771885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe321b9-5156-4ef4-a0af-0fb3e1e1f82e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:18.783374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe321b9-5156-4ef4-a0af-0fb3e1e1f82e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:18.787186 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e457be-d0c3-4ad5-8987-2d092cade6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:18.787791 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e457be-d0c3-4ad5-8987-2d092cade6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:18.788099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e457be-d0c3-4ad5-8987-2d092cade6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:18.788373 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e457be-d0c3-4ad5-8987-2d092cade6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:18.854538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e457be-d0c3-4ad5-8987-2d092cade6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:18.855977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e457be-d0c3-4ad5-8987-2d092cade6a4 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:18.865148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1260/2507] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:21.223434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:21.225017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.283404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c692c9c-ad47-46cf-854e-d6b48cf93456 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:21.287496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00189fb-966f-4cce-848a-66affcb442d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:21.288313 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00189fb-966f-4cce-848a-66affcb442d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.288773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00189fb-966f-4cce-848a-66affcb442d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.288922 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00189fb-966f-4cce-848a-66affcb442d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.290802 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d00189fb-966f-4cce-848a-66affcb442d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.301142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00189fb-966f-4cce-848a-66affcb442d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:21.303117 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00189fb-966f-4cce-848a-66affcb442d9 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:21.312418 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1248/2508] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:07:21.490891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345460f-bbba-45a1-b99d-0aa898b20472 None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.491367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345460f-bbba-45a1-b99d-0aa898b20472 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.491513 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345460f-bbba-45a1-b99d-0aa898b20472 None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.493486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1261/2509] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:07:21.500058 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.500345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.500646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.536925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:21.540219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.551023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:21.556065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345460f-bbba-45a1-b99d-0aa898b20472 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:21.660290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345460f-bbba-45a1-b99d-0aa898b20472 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:21.665334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:21.666101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.666434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.666721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] PATH_INFO: `/v3/users/00c14a8bc2014271b2b3478284eee189` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.667950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c14a8bc2014271b2b3478284eee189)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:21.670224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:21.693800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1249/2510] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:07:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:21.714160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:21.715983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.763103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c14a8bc2014271b2b3478284eee189'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:21.788530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c14a8bc2014271b2b3478284eee189'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:21.788530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c14a8bc2014271b2b3478284eee189'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:21.788530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c14a8bc2014271b2b3478284eee189'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:21.800253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:21.802238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f39efc37-fb75-4f06-bfaa-942703912d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c14a8bc2014271b2b3478284eee189'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:21.804354 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5738-20c0-4149-a618-de5f388a194f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:21.808123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cba0f-25bb-463e-9494-a929d61ae2c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:21.808771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cba0f-25bb-463e-9494-a929d61ae2c2 service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.809050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cba0f-25bb-463e-9494-a929d61ae2c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.809323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cba0f-25bb-463e-9494-a929d61ae2c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.811237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4c7cba0f-25bb-463e-9494-a929d61ae2c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.815859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1262/2511] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:21 2022] DELETE /identity/v3/users/00c14a8bc2014271b2b3478284eee189 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:21.817346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.817700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.818040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.825501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cba0f-25bb-463e-9494-a929d61ae2c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:21.826973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cba0f-25bb-463e-9494-a929d61ae2c2 service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:21.837976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:21.838358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1250/2512] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:21.839547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.904271 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1263/2513] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:21.908368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6aef-053c-4a14-8e48-90273874ab83 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.908638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6aef-053c-4a14-8e48-90273874ab83 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.908914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6aef-053c-4a14-8e48-90273874ab83 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.910420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1251/2514] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:07:21.914384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:21.914701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131838f-f69b-4e14-8e32-d6f803ab24c8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.915055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131838f-f69b-4e14-8e32-d6f803ab24c8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.915398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131838f-f69b-4e14-8e32-d6f803ab24c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.915783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.934848 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a131838f-f69b-4e14-8e32-d6f803ab24c8 None None] MFA Rules not processed for user `a4b10367002d4f64a82b3e9eca00ca06`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:21.936156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a131838f-f69b-4e14-8e32-d6f803ab24c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:21.973403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f402b-a09d-4891-ab77-53359e4b98b3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:21.977178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4c14-8bd7-45e6-b26a-4ac1edca1705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:21.977812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4c14-8bd7-45e6-b26a-4ac1edca1705 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:21.978097 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4c14-8bd7-45e6-b26a-4ac1edca1705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:21.978359 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4c14-8bd7-45e6-b26a-4ac1edca1705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:21.987705 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4c14-8bd7-45e6-b26a-4ac1edca1705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:21.988330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1252/2515] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:07:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:21.989284 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4c14-8bd7-45e6-b26a-4ac1edca1705 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:21.999439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1264/2516] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:07:22.000663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131838f-f69b-4e14-8e32-d6f803ab24c8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:22.010358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131838f-f69b-4e14-8e32-d6f803ab24c8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:22.014761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee73a4-954c-4397-a8ab-797fadad67be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:22.015386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee73a4-954c-4397-a8ab-797fadad67be service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:22.015655 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee73a4-954c-4397-a8ab-797fadad67be service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:22.015927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee73a4-954c-4397-a8ab-797fadad67be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:22.023262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee73a4-954c-4397-a8ab-797fadad67be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:22.025201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee73a4-954c-4397-a8ab-797fadad67be service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:22.034380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1253/2517] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:22.136256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3300-c67e-4b7b-b1b2-0ad10a06b99f None None] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:22.136571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3300-c67e-4b7b-b1b2-0ad10a06b99f None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:22.136844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3300-c67e-4b7b-b1b2-0ad10a06b99f None None] PATH_INFO: `/` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:22.138365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1265/2518] 10.222.0.43 () {54 vars in 921 bytes} [Tue Sep 13 18:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 18:07:22.144075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:22.144359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:22.144662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:22.165127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] MFA Rules not processed for user `90f40fe8f20f4a5bbd201debda6a97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:22.166273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:22.208643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:22.245582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1254/2519] 10.222.0.43 () {58 vars in 1018 bytes} [Tue Sep 13 18:07:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:22.257035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3300-c67e-4b7b-b1b2-0ad10a06b99f None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:22.269283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3300-c67e-4b7b-b1b2-0ad10a06b99f None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:22.274025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:22.274630 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:22.274905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:22.275169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:22.287612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:22.289111 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:22.299731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1266/2520] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:22.329832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:22.331446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:22.346326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fbeb9-8ffa-4ed0-8d31-f621d5d9f050 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:22.360926 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:22.361614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:22.362083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:22.362545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] PATH_INFO: `/v3/projects/2c2939d1d73e44c6bf86b4adfe781a4c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:22.363887 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2939d1d73e44c6bf86b4adfe781a4c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:22.366177 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:22.419352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2939d1d73e44c6bf86b4adfe781a4c', 'request_id': 'req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.427333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2939d1d73e44c6bf86b4adfe781a4c', 'request_id': 'req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.434137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2939d1d73e44c6bf86b4adfe781a4c', 'request_id': 'req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.445305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] The token cache is being invalidate because project 2c2939d1d73e44c6bf86b4adfe781a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:22.445623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.446439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.448204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1255/2521] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:22 2022] DELETE /identity/v3/projects/2c2939d1d73e44c6bf86b4adfe781a4c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:22.452516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:22.464188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b27d9-7268-4e51-8c15-7e93543134ba service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:22.528083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:22.528812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:22.529023 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:22.529301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] PATH_INFO: `/v3/users/5067f308ecbb4b3bbd8e780c8fa1f621` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:22.530449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5067f308ecbb4b3bbd8e780c8fa1f621)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:22.532804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:22.565015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:22.585904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5067f308ecbb4b3bbd8e780c8fa1f621'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.596465 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5067f308ecbb4b3bbd8e780c8fa1f621'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.597309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5067f308ecbb4b3bbd8e780c8fa1f621'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.604451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5067f308ecbb4b3bbd8e780c8fa1f621'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.617155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e34b693c-5893-49e8-9315-cb5670a61709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5067f308ecbb4b3bbd8e780c8fa1f621'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:22.623299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1267/2522] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:22 2022] DELETE /identity/v3/users/5067f308ecbb4b3bbd8e780c8fa1f621 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:22.962189 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:22.963702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:22.984079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf21fcf-a3e2-49c8-94c7-cb2ef48807d4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.062648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.063491 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.064524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.064804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] PATH_INFO: `/v3/projects/a93940372597409688fc94426f6b7670` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.066110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93940372597409688fc94426f6b7670)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.069382 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.125288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93940372597409688fc94426f6b7670', 'request_id': 'req-10a270aa-20db-49d6-8a47-9fa87ee089a2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:23.133505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93940372597409688fc94426f6b7670', 'request_id': 'req-10a270aa-20db-49d6-8a47-9fa87ee089a2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:23.134670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:23.142023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93940372597409688fc94426f6b7670', 'request_id': 'req-10a270aa-20db-49d6-8a47-9fa87ee089a2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:23.161143 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] The token cache is being invalidate because project a93940372597409688fc94426f6b7670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:23.161281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:23.161950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10a270aa-20db-49d6-8a47-9fa87ee089a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:23.163454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1256/2523] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:22 2022] DELETE /identity/v3/projects/a93940372597409688fc94426f6b7670 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:23.168401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068b4b-46db-43fd-9401-f6bc5b1d6055 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.168728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068b4b-46db-43fd-9401-f6bc5b1d6055 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.169003 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068b4b-46db-43fd-9401-f6bc5b1d6055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.188992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-51068b4b-46db-43fd-9401-f6bc5b1d6055 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:23.190191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-51068b4b-46db-43fd-9401-f6bc5b1d6055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:23.257212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1268/2524] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:23.265279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4132-6d35-4112-94ab-74556d91ba20 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.265747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4132-6d35-4112-94ab-74556d91ba20 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.265861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4132-6d35-4112-94ab-74556d91ba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.285572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-efcc4132-6d35-4112-94ab-74556d91ba20 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:23.286786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-efcc4132-6d35-4112-94ab-74556d91ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:23.295636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1257/2525] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:23.299933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51068b4b-46db-43fd-9401-f6bc5b1d6055 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.311754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51068b4b-46db-43fd-9401-f6bc5b1d6055 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.318657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.319304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.319591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.319858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.320575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:07:23.320863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.322495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.335873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1269/2526] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:07:23.340471 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4132-6d35-4112-94ab-74556d91ba20 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.350536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4132-6d35-4112-94ab-74556d91ba20 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.355475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.356092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.356368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.356763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.357423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.359357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.399732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1258/2527] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:23 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:07:23.402192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.416897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7ea9-c648-45a3-884c-eb611130e03b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.496611 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.497345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.497698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.498041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.498761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.501355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.531959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1270/2528] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:23 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:07:23.534270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.545844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a322a-95b9-4cc0-9211-21beb74e0980 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.550956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.551668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.551822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.552098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.552775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.554380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.567814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1259/2529] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:23.569123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.579778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a9d7b-6ab0-4c89-b56b-2fcc936c034c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.584415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bad35-fb06-4128-a381-6e1b88ac95d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.585047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bad35-fb06-4128-a381-6e1b88ac95d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.585322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bad35-fb06-4128-a381-6e1b88ac95d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.585587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bad35-fb06-4128-a381-6e1b88ac95d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.586235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bad35-fb06-4128-a381-6e1b88ac95d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.587896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94bad35-fb06-4128-a381-6e1b88ac95d4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.601446 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1271/2530] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:23.605964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.617708 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb61fd7-e81c-4b6f-946a-34614f786424 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.623325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.624180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.624594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.625107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] PATH_INFO: `/v3/projects/0f2ce33bbb8d4b1cacbb3c37f71c5f3e/users/768a0decbdc943c5b8c74031b74a8d1d/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.642067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2ce33bbb8d4b1cacbb3c37f71c5f3e, user_id=768a0decbdc943c5b8c74031b74a8d1d, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.644960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.660165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1260/2531] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:23 2022] PUT /identity/v3/projects/0f2ce33bbb8d4b1cacbb3c37f71c5f3e/users/768a0decbdc943c5b8c74031b74a8d1d/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:23.672088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a0596-d966-43dc-9960-c4533eda55cb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.672385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a0596-d966-43dc-9960-c4533eda55cb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.672681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a0596-d966-43dc-9960-c4533eda55cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.694052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:23.702591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-976a0596-d966-43dc-9960-c4533eda55cb None None] MFA Rules not processed for user `768a0decbdc943c5b8c74031b74a8d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:23.704396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-976a0596-d966-43dc-9960-c4533eda55cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:23.771155 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1272/2532] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:23.775817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.837814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c322814-068f-4a4f-9e09-8a73636c8c09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.842908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.843510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.843783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.844048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.844714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.846609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.876164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1261/2533] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:23 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:23.880935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a0596-d966-43dc-9960-c4533eda55cb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:23.893561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a0596-d966-43dc-9960-c4533eda55cb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:23.964722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:23.965454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:23.965808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:23.966163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:23.967046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:23.969811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:23.993467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1273/2534] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:23 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:23.997668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:24.009410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbe305-3ac8-4740-a039-bee3a4b79566 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:24.015214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:24.015959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.016339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.016740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.017557 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:24.019360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:24.032067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1262/2535] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:07:24.036595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:24.048729 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30fa03-2cf8-43cf-b401-fa506d2be07c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:24.055505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:24.056346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.056788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.057171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] PATH_INFO: `/v3/projects/6eb435cd03e147009a113d763ff43018/users/733ad96d93154e12bd3cd06b81298b23/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.076079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb435cd03e147009a113d763ff43018, user_id=733ad96d93154e12bd3cd06b81298b23, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:24.078776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:24.096036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1274/2536] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:24 2022] PUT /identity/v3/projects/6eb435cd03e147009a113d763ff43018/users/733ad96d93154e12bd3cd06b81298b23/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:07:24.100345 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:24.112225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd01428-6ca9-4da7-b968-3e0cbcad6377 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:24.165699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:24.175263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1886994-8c14-4637-9099-061ed7286050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:24.176081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1886994-8c14-4637-9099-061ed7286050 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.176556 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1886994-8c14-4637-9099-061ed7286050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.176978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1886994-8c14-4637-9099-061ed7286050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.178513 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1886994-8c14-4637-9099-061ed7286050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:24.179876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1886994-8c14-4637-9099-061ed7286050 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:24.190218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1263/2537] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:07:24.194718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:24.206068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d266e-4d5d-4255-bb20-8acc3ebe22eb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:24.210861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:24.211517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.211803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.212110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] PATH_INFO: `/v3/projects/6eb435cd03e147009a113d763ff43018/users/733ad96d93154e12bd3cd06b81298b23/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.214088 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb435cd03e147009a113d763ff43018, user_id=733ad96d93154e12bd3cd06b81298b23, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:24.217194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:24.230963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1275/2538] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:24 2022] PUT /identity/v3/projects/6eb435cd03e147009a113d763ff43018/users/733ad96d93154e12bd3cd06b81298b23/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:24.241520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386474b-1395-404d-9527-789c419cb2bc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.241808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386474b-1395-404d-9527-789c419cb2bc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.242115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386474b-1395-404d-9527-789c419cb2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.276883 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7386474b-1395-404d-9527-789c419cb2bc None None] MFA Rules not processed for user `733ad96d93154e12bd3cd06b81298b23`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:24.278124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7386474b-1395-404d-9527-789c419cb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:24.351157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1264/2539] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:24.362414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:24.364198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:24.383548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1d996-a36f-47c7-b5eb-bb81f2111615 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.383838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1d996-a36f-47c7-b5eb-bb81f2111615 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.384115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1d996-a36f-47c7-b5eb-bb81f2111615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.403814 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-20e1d996-a36f-47c7-b5eb-bb81f2111615 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:24.405087 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-20e1d996-a36f-47c7-b5eb-bb81f2111615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:24.423272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52cbd7-962d-422a-9690-fbbf5089ce9e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:24.426986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:24.427618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.427892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.428311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.435264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:24.437764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:24.439440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:24.450531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1276/2540] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:24.463082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1265/2541] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:07:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:24.475388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:24.477362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:24.548713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc0b8a-d8c9-41c8-8a09-b3ed7cd10ca1 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:24.554261 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c91a3b-26d9-47ae-9f0f-8618016076f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:24.554877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c91a3b-26d9-47ae-9f0f-8618016076f8 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:24.555160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c91a3b-26d9-47ae-9f0f-8618016076f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:24.555429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c91a3b-26d9-47ae-9f0f-8618016076f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:24.564009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c91a3b-26d9-47ae-9f0f-8618016076f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:24.565619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c91a3b-26d9-47ae-9f0f-8618016076f8 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:24.573724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1277/2542] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:07:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:07:25.171628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1d996-a36f-47c7-b5eb-bb81f2111615 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:25.294067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1d996-a36f-47c7-b5eb-bb81f2111615 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:25.306009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:25.306725 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:25.307062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:25.307404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] PATH_INFO: `/v3/users/768a0decbdc943c5b8c74031b74a8d1d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:25.308924 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] RBAC: Authorizing `identity:delete_user(user_id=768a0decbdc943c5b8c74031b74a8d1d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:25.311272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:25.348839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:25.392555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '768a0decbdc943c5b8c74031b74a8d1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:25.405508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '768a0decbdc943c5b8c74031b74a8d1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:25.406654 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '768a0decbdc943c5b8c74031b74a8d1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:25.417735 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '768a0decbdc943c5b8c74031b74a8d1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:25.433048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '768a0decbdc943c5b8c74031b74a8d1d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:25.442118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1266/2543] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:25 2022] DELETE /identity/v3/users/768a0decbdc943c5b8c74031b74a8d1d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:25.446904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:25.447195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:25.447471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:25.468936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:25.470487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:25.542015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1278/2544] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:25.551979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:25.553479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:25.613561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea104bb5-cfc3-488e-b8f4-13acd723808e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:25.618190 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:25.618885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:25.619254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:25.619580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:25.629748 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:25.631232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:25.642044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1267/2545] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:25.942582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:25.944121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:25.957854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a7f46-775d-44cc-9159-bece2b34f725 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:25.964126 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:25.965277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:25.965746 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:25.966216 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] PATH_INFO: `/v3/projects/0f2ce33bbb8d4b1cacbb3c37f71c5f3e` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:25.967627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2ce33bbb8d4b1cacbb3c37f71c5f3e)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:25.970080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:26.033164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2ce33bbb8d4b1cacbb3c37f71c5f3e', 'request_id': 'req-33f91110-9617-4656-b396-79676fdba570'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.041054 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2ce33bbb8d4b1cacbb3c37f71c5f3e', 'request_id': 'req-33f91110-9617-4656-b396-79676fdba570'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.047173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2ce33bbb8d4b1cacbb3c37f71c5f3e', 'request_id': 'req-33f91110-9617-4656-b396-79676fdba570'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.058618 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] The token cache is being invalidate because project 0f2ce33bbb8d4b1cacbb3c37f71c5f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:26.058920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.059584 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.064191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1279/2546] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:25 2022] DELETE /identity/v3/projects/0f2ce33bbb8d4b1cacbb3c37f71c5f3e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:26.065326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:26.075833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47382ac-160b-4e7c-8af1-ec6080f57dd0 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:26.141722 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:26.142416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:26.142684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:26.142949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] PATH_INFO: `/v3/users/733ad96d93154e12bd3cd06b81298b23` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:26.144055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=733ad96d93154e12bd3cd06b81298b23)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:26.146020 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:26.198555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:26.200203 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733ad96d93154e12bd3cd06b81298b23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.209057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733ad96d93154e12bd3cd06b81298b23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.209914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733ad96d93154e12bd3cd06b81298b23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.218481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733ad96d93154e12bd3cd06b81298b23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.233142 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-81093d06-94c8-46ca-aca9-eccb607c9348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733ad96d93154e12bd3cd06b81298b23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.239998 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1268/2547] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:26 2022] DELETE /identity/v3/users/733ad96d93154e12bd3cd06b81298b23 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:26.649022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:26.650520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:26.678956 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f91110-9617-4656-b396-79676fdba570 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:26.692753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:26.765074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:26.765754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:26.766042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:26.766312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] PATH_INFO: `/v3/projects/6eb435cd03e147009a113d763ff43018` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:26.767439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb435cd03e147009a113d763ff43018)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:26.769420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:26.826435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb435cd03e147009a113d763ff43018', 'request_id': 'req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.834617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb435cd03e147009a113d763ff43018', 'request_id': 'req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.845571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb435cd03e147009a113d763ff43018', 'request_id': 'req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.858688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] The token cache is being invalidate because project 6eb435cd03e147009a113d763ff43018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:26.858991 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.859679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d2aab9ef-c04e-4738-8cbf-0b14fd7a89e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:26.861482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1280/2548] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:26 2022] DELETE /identity/v3/projects/6eb435cd03e147009a113d763ff43018 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:26.866386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ac1ba-54cd-4add-8aa6-0358b3790b90 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:26.866971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ac1ba-54cd-4add-8aa6-0358b3790b90 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:26.866971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ac1ba-54cd-4add-8aa6-0358b3790b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:26.898765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-aa6ac1ba-54cd-4add-8aa6-0358b3790b90 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:26.900637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aa6ac1ba-54cd-4add-8aa6-0358b3790b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:26.977481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1269/2549] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:26.986855 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fd30-ac11-40e0-9a35-9ed88ce2885e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:26.987438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fd30-ac11-40e0-9a35-9ed88ce2885e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:26.987438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fd30-ac11-40e0-9a35-9ed88ce2885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.013292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ca17fd30-ac11-40e0-9a35-9ed88ce2885e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:27.014524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ca17fd30-ac11-40e0-9a35-9ed88ce2885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:27.022436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1281/2550] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:27.027009 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ac1ba-54cd-4add-8aa6-0358b3790b90 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.037103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ac1ba-54cd-4add-8aa6-0358b3790b90 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.042014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.042626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.042902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.043204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.043871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:07:27.044187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.045867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.058287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1270/2551] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:07:27.062835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fd30-ac11-40e0-9a35-9ed88ce2885e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.073020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fd30-ac11-40e0-9a35-9ed88ce2885e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.077961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.078766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.079040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.079315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.079981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.082007 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.115488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1282/2552] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:27 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:07:27.119325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.130531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86848cc-9391-40af-8542-a2e402627c0c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.191301 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.191850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.192120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.192388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.193078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.196541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.226434 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1271/2553] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:27 2022] POST /identity/v3/users => generated 281 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:27.230563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.244286 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd5e59-bf11-43a4-af6f-4abf86237513 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.250920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.251541 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.251811 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.252468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.253116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.254711 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.271850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1283/2554] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:27.276083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.289000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c23ac-5aeb-416c-8c70-a6a8c87612fe admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.293927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72a24c-9001-41ae-816a-e60f6e23c641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.294555 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72a24c-9001-41ae-816a-e60f6e23c641 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.294829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72a24c-9001-41ae-816a-e60f6e23c641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.295097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72a24c-9001-41ae-816a-e60f6e23c641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.295751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72a24c-9001-41ae-816a-e60f6e23c641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.297487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72a24c-9001-41ae-816a-e60f6e23c641 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.309229 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1272/2555] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:27.313748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.325141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3927f18-c6ca-4023-9427-d647e31fdf95 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.330962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.331716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.332279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.332699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] PATH_INFO: `/v3/projects/56b58478cf22400c8fa3a6cc22c84e92/users/30ff9c66a797466992773b1a069aa6c8/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.349038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b58478cf22400c8fa3a6cc22c84e92, user_id=30ff9c66a797466992773b1a069aa6c8, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.351730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.367628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1284/2556] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:27 2022] PUT /identity/v3/projects/56b58478cf22400c8fa3a6cc22c84e92/users/30ff9c66a797466992773b1a069aa6c8/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:27.379349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320edae-79c1-4b7a-937a-6cab82a1a2e0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.379636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320edae-79c1-4b7a-937a-6cab82a1a2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.379909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320edae-79c1-4b7a-937a-6cab82a1a2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.411222 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b320edae-79c1-4b7a-937a-6cab82a1a2e0 None None] MFA Rules not processed for user `30ff9c66a797466992773b1a069aa6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:27.412490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b320edae-79c1-4b7a-937a-6cab82a1a2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:27.433540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:27.485614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1273/2557] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:27.520714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.522688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:27.590240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:27.613092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d4a0d-24e8-43c1-8828-20813193809f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.618178 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2e741-c7f8-400b-845f-d7228a7bfea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.619265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2e741-c7f8-400b-845f-d7228a7bfea5 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.619759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2e741-c7f8-400b-845f-d7228a7bfea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.620281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2e741-c7f8-400b-845f-d7228a7bfea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.630062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2e741-c7f8-400b-845f-d7228a7bfea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.631565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2e741-c7f8-400b-845f-d7228a7bfea5 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.641303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1285/2558] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:27.656402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320edae-79c1-4b7a-937a-6cab82a1a2e0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.754320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320edae-79c1-4b7a-937a-6cab82a1a2e0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:27.769981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:27.770700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.770983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.771257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] PATH_INFO: `/v3/users/30ff9c66a797466992773b1a069aa6c8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.772585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ff9c66a797466992773b1a069aa6c8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:27.774723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:27.843285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ff9c66a797466992773b1a069aa6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:27.855727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ff9c66a797466992773b1a069aa6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:27.857023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ff9c66a797466992773b1a069aa6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:27.874958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ff9c66a797466992773b1a069aa6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:27.890183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ff9c66a797466992773b1a069aa6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:27.899990 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1274/2559] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:27 2022] DELETE /identity/v3/users/30ff9c66a797466992773b1a069aa6c8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:27.901438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:27.901725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:27.902000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:27.922848 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:27.924077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:27.986730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1286/2560] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:27.997010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:27.998533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:28.058497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:28.076339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7bc6d-f122-4cf9-b3d0-262b124e4fd5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:28.080843 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510728b-b72b-4f4f-80a1-2c4fda9cdd11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:28.081605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510728b-b72b-4f4f-80a1-2c4fda9cdd11 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.081987 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510728b-b72b-4f4f-80a1-2c4fda9cdd11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.082297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510728b-b72b-4f4f-80a1-2c4fda9cdd11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.094156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510728b-b72b-4f4f-80a1-2c4fda9cdd11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:28.095669 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510728b-b72b-4f4f-80a1-2c4fda9cdd11 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:28.106498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1275/2561] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:28.424440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:28.426331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:28.449312 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4477c2-2255-45f9-b44c-95294ccbc653 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:28.455433 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:28.456068 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.456846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.457134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] PATH_INFO: `/v3/projects/56b58478cf22400c8fa3a6cc22c84e92` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.458365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b58478cf22400c8fa3a6cc22c84e92)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:28.460955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:28.514379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b58478cf22400c8fa3a6cc22c84e92', 'request_id': 'req-73a66e99-6728-4434-b1a6-2465afdc8424'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:28.525676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b58478cf22400c8fa3a6cc22c84e92', 'request_id': 'req-73a66e99-6728-4434-b1a6-2465afdc8424'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:28.531960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b58478cf22400c8fa3a6cc22c84e92', 'request_id': 'req-73a66e99-6728-4434-b1a6-2465afdc8424'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:28.543237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] The token cache is being invalidate because project 56b58478cf22400c8fa3a6cc22c84e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:28.543530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:28.544156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-73a66e99-6728-4434-b1a6-2465afdc8424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:28.545683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1287/2562] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:28 2022] DELETE /identity/v3/projects/56b58478cf22400c8fa3a6cc22c84e92 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:28.551250 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8ee8e-f26b-4289-b331-b6d84b0266c0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.551531 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8ee8e-f26b-4289-b331-b6d84b0266c0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.551825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8ee8e-f26b-4289-b331-b6d84b0266c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.582897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-07f8ee8e-f26b-4289-b331-b6d84b0266c0 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:28.584038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-07f8ee8e-f26b-4289-b331-b6d84b0266c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:28.669794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1276/2563] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:28.672005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b77505-a012-4aa6-838c-cd3d0dd0479d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.672592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b77505-a012-4aa6-838c-cd3d0dd0479d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.673078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b77505-a012-4aa6-838c-cd3d0dd0479d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.696092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-06b77505-a012-4aa6-838c-cd3d0dd0479d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:28.697697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-06b77505-a012-4aa6-838c-cd3d0dd0479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:28.706554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1288/2564] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:28.710955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8ee8e-f26b-4289-b331-b6d84b0266c0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:28.721502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8ee8e-f26b-4289-b331-b6d84b0266c0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:28.726269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:28.726886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.727167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.727462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.728132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:07:28.728442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:28.730306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:28.743831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1277/2565] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:07:28.748305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b77505-a012-4aa6-838c-cd3d0dd0479d None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:28.758603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b77505-a012-4aa6-838c-cd3d0dd0479d None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:28.763731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:28.764700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.765085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.765520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.766497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:28.777743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:28.804551 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:28.804794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1289/2566] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:28 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:28.817842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac92a82-8ce2-4195-9ec2-e26ba8cf5362 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:28.877393 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:28.878850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.879135 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.879408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.880261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:28.882276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:28.908218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1278/2567] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:28 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:07:28.910415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:28.924745 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c20bcf-0e17-42f2-8319-4200cae735ed admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:28.933217 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:28.933943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.934344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.934632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.935364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:28.944468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:28.955250 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1290/2568] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:07:28.959408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:28.969892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0f404-c7c0-4017-a74b-caf88d3f5e5b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:28.985291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78088dfd-abec-42ed-a732-82492d93879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:28.986195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78088dfd-abec-42ed-a732-82492d93879b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:28.986575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78088dfd-abec-42ed-a732-82492d93879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:28.986950 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78088dfd-abec-42ed-a732-82492d93879b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:28.987733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78088dfd-abec-42ed-a732-82492d93879b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:28.989823 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78088dfd-abec-42ed-a732-82492d93879b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.003530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1279/2569] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:29.008001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.020639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14001964-7ff3-4f14-8fdb-ff2c4e417df3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.025757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.026589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.027309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.027309 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] PATH_INFO: `/v3/projects/643c5dd88b1348b1b501a953d7699b05/users/a2415563dfed4e92875f1d80827e40ab/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.062789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643c5dd88b1348b1b501a953d7699b05, user_id=a2415563dfed4e92875f1d80827e40ab, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.065954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.078089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:29.103812 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1291/2570] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:29 2022] PUT /identity/v3/projects/643c5dd88b1348b1b501a953d7699b05/users/a2415563dfed4e92875f1d80827e40ab/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:29.107752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22939d-e393-4546-a839-45b055d6b9cc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.108041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22939d-e393-4546-a839-45b055d6b9cc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.108312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22939d-e393-4546-a839-45b055d6b9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.138635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4e22939d-e393-4546-a839-45b055d6b9cc None None] MFA Rules not processed for user `a2415563dfed4e92875f1d80827e40ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:29.139772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e22939d-e393-4546-a839-45b055d6b9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:29.175795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:29.212291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1280/2571] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:29.216766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.282488 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91308aca-7857-4e27-a923-725aa45326c2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.287209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.287820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.288101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.288376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.289051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.291009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.328875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1292/2572] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:29 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:29.334329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22939d-e393-4546-a839-45b055d6b9cc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.346643 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22939d-e393-4546-a839-45b055d6b9cc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.407389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.407939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.408283 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.408625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.409305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.411329 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.438573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1281/2573] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:29 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:29.442761 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.453802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d73385-b090-4c51-a072-f66f685dde0d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.458909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.459521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.459794 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.460061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.460737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.462335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.475220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1293/2574] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:29.479762 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.493750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356342-38ce-482f-9607-a6c0561fd613 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.498831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872a5c8-b25e-47f8-971c-429f6b347e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.499461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a5c8-b25e-47f8-971c-429f6b347e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.499742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a5c8-b25e-47f8-971c-429f6b347e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.500016 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a5c8-b25e-47f8-971c-429f6b347e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.501634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872a5c8-b25e-47f8-971c-429f6b347e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.503721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872a5c8-b25e-47f8-971c-429f6b347e76 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.516462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1282/2575] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:29.519531 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.531161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7c3d3-f18d-4ca7-9d52-d80d628547f8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.535751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.536441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.536715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.537002 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] PATH_INFO: `/v3/projects/cd78b4ee48774a17aadf1b1ecb6c9a53/users/541b63a18c664a2bbe9edbcb94019cde/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.553969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd78b4ee48774a17aadf1b1ecb6c9a53, user_id=541b63a18c664a2bbe9edbcb94019cde, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.556765 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.576792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1294/2576] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:29 2022] PUT /identity/v3/projects/cd78b4ee48774a17aadf1b1ecb6c9a53/users/541b63a18c664a2bbe9edbcb94019cde/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:29.650449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66db9f-0d41-41ba-97e1-9c284ec0deff None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.650834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66db9f-0d41-41ba-97e1-9c284ec0deff None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.651116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66db9f-0d41-41ba-97e1-9c284ec0deff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.684359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6d66db9f-0d41-41ba-97e1-9c284ec0deff None None] MFA Rules not processed for user `541b63a18c664a2bbe9edbcb94019cde`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:29.685622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6d66db9f-0d41-41ba-97e1-9c284ec0deff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:29.768569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1283/2577] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:29.773277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.837717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091b1f1-ce98-4b33-9805-707c9bdec306 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.842990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.843753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.844152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.844527 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.845285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.847265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.875793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1295/2578] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:29 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:29.880591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66db9f-0d41-41ba-97e1-9c284ec0deff None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:29.898921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66db9f-0d41-41ba-97e1-9c284ec0deff None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:29.950550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:29.967188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:29.967809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:29.968088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:29.968360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:29.969039 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:29.970984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:29.996179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1284/2579] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:29 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:30.000135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:30.011557 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fd3c1-fbf5-44ab-ada8-52bdcc68604c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:30.017102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:30.017694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:30.017969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:30.018233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:30.018885 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:30.020543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:30.032474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1296/2580] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:30.036769 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:30.048453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5177e40-b33c-4463-861b-8703039a9f42 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:30.053249 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:30.053921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:30.054196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:30.054469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] PATH_INFO: `/v3/projects/82d149f12c64447b8b51ec4308c9fca6/users/b66d4035812645faadc455185be0a760/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:30.069684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d149f12c64447b8b51ec4308c9fca6, user_id=b66d4035812645faadc455185be0a760, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:30.072344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:30.086155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1285/2581] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:30 2022] PUT /identity/v3/projects/82d149f12c64447b8b51ec4308c9fca6/users/b66d4035812645faadc455185be0a760/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:30.090448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:30.101077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92d4e0-137d-4305-b5e0-51c90a7f66d1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:30.126563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:30.166675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea9276-1788-4dd1-bad6-ef28c480c2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:30.167507 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea9276-1788-4dd1-bad6-ef28c480c2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:30.167865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea9276-1788-4dd1-bad6-ef28c480c2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:30.168375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea9276-1788-4dd1-bad6-ef28c480c2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:30.169139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea9276-1788-4dd1-bad6-ef28c480c2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:30.170784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea9276-1788-4dd1-bad6-ef28c480c2c9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:30.181380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1297/2582] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:30.185391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:30.196374 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a0bbb-469e-41a8-8871-9c83cef4ed4c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:30.201889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:30.202496 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:30.202773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:30.203050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] PATH_INFO: `/v3/projects/82d149f12c64447b8b51ec4308c9fca6/users/b66d4035812645faadc455185be0a760/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:30.205495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d149f12c64447b8b51ec4308c9fca6, user_id=b66d4035812645faadc455185be0a760, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:30.208298 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:30.220172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1286/2583] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:30 2022] PUT /identity/v3/projects/82d149f12c64447b8b51ec4308c9fca6/users/b66d4035812645faadc455185be0a760/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:07:30.230785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:30.231124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:30.231432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:30.260854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] MFA Rules not processed for user `b66d4035812645faadc455185be0a760`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:30.262471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:30.338289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1298/2584] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:30.351390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:30.353256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:30.417608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29a98f-3414-4c87-9df4-7f1d4b1cd1bc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:30.422205 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:30.423156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:30.423628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:30.423997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:30.484339 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:30.485909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:30.502746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1287/2585] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:07:32.085792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:32.087627 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:32.108020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cec36-25fd-4026-a349-5d0051b534bb None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:32.112701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:32.113757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:32.114041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:32.114317 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:32.177214 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:32.178684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:32.189709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1299/2586] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:32.825363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:32.834652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5fd2a-ac5b-4c38-964c-f64a7611fba8 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:32.839019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:32.839666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:32.839957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:32.840206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:32.850107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:32.851586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:32.861188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1288/2587] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:35.013707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:35.015748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:35.107848 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:35.144568 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b8e7-2d45-4296-9a96-7048168f10c6 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:35.151057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887d2dc-b264-4485-8da9-7c8efe4f8ecb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:35.152080 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d2dc-b264-4485-8da9-7c8efe4f8ecb service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:35.152693 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d2dc-b264-4485-8da9-7c8efe4f8ecb service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:35.153267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d2dc-b264-4485-8da9-7c8efe4f8ecb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:35.156475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2887d2dc-b264-4485-8da9-7c8efe4f8ecb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:35.188331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887d2dc-b264-4485-8da9-7c8efe4f8ecb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:35.190034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887d2dc-b264-4485-8da9-7c8efe4f8ecb service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:35.203236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1300/2588] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:36.846252 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:36.886978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:36.909445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da7a37-5776-450f-8332-1b9b9de460cd service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:36.914235 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:36.914863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:36.915139 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:36.915394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] PATH_INFO: `/v3/users/a2415563dfed4e92875f1d80827e40ab` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:36.916570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2415563dfed4e92875f1d80827e40ab)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:36.918573 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:36.968796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2415563dfed4e92875f1d80827e40ab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:36.978325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2415563dfed4e92875f1d80827e40ab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:36.978967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2415563dfed4e92875f1d80827e40ab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:36.986456 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2415563dfed4e92875f1d80827e40ab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:36.998662 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2415563dfed4e92875f1d80827e40ab'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.005854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1289/2589] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:36 2022] DELETE /identity/v3/users/a2415563dfed4e92875f1d80827e40ab => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:37.010320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915edd95-bd06-4745-a915-1e688644c218 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:37.011190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915edd95-bd06-4745-a915-1e688644c218 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:37.011190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915edd95-bd06-4745-a915-1e688644c218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:37.030612 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-915edd95-bd06-4745-a915-1e688644c218 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:37.031834 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-915edd95-bd06-4745-a915-1e688644c218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:37.092538 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1301/2590] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:07:37.102718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:37.104348 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:37.169697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa180b1-e75a-4f31-88b8-6d3f6c086142 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:37.175481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:37.176154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:37.176445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:37.177075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:37.187908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:37.189490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:37.204609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1290/2591] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:37.507580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915edd95-bd06-4745-a915-1e688644c218 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:37.509353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-915edd95-bd06-4745-a915-1e688644c218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:37.527448 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915edd95-bd06-4745-a915-1e688644c218 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:37.532578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:37.533776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:37.534278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:37.534844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] PATH_INFO: `/v3/projects/643c5dd88b1348b1b501a953d7699b05` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:37.536161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] RBAC: Authorizing `identity:delete_project(project_id=643c5dd88b1348b1b501a953d7699b05)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:37.539607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:37.598160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643c5dd88b1348b1b501a953d7699b05', 'request_id': 'req-9558d610-96ce-4a8e-bf02-efd57c2c08be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.608418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643c5dd88b1348b1b501a953d7699b05', 'request_id': 'req-9558d610-96ce-4a8e-bf02-efd57c2c08be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.616117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643c5dd88b1348b1b501a953d7699b05', 'request_id': 'req-9558d610-96ce-4a8e-bf02-efd57c2c08be'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.629363 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] The token cache is being invalidate because project 643c5dd88b1348b1b501a953d7699b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:37.629759 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.630342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.632104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1302/2592] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:37 2022] DELETE /identity/v3/projects/643c5dd88b1348b1b501a953d7699b05 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:37.636012 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:37.645871 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a050df7-e31f-4bd1-9ae6-fd43570e0e3d service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:37.763276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:37.764072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:37.764490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:37.764876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] PATH_INFO: `/v3/users/541b63a18c664a2bbe9edbcb94019cde` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:37.766297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] RBAC: Authorizing `identity:delete_user(user_id=541b63a18c664a2bbe9edbcb94019cde)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:37.769383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:37.770718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:37.838196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541b63a18c664a2bbe9edbcb94019cde'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.856517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541b63a18c664a2bbe9edbcb94019cde'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.859809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541b63a18c664a2bbe9edbcb94019cde'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.874157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541b63a18c664a2bbe9edbcb94019cde'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.888182 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541b63a18c664a2bbe9edbcb94019cde'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:37.898270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1291/2593] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:37 2022] DELETE /identity/v3/users/541b63a18c664a2bbe9edbcb94019cde => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:07:38.330474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:38.333139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:38.347650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558d610-96ce-4a8e-bf02-efd57c2c08be admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:38.415109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:38.426676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:38.427324 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:38.427607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:38.427877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] PATH_INFO: `/v3/projects/cd78b4ee48774a17aadf1b1ecb6c9a53` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:38.429047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd78b4ee48774a17aadf1b1ecb6c9a53)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:38.430959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:38.478602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd78b4ee48774a17aadf1b1ecb6c9a53', 'request_id': 'req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.487391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd78b4ee48774a17aadf1b1ecb6c9a53', 'request_id': 'req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.493243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd78b4ee48774a17aadf1b1ecb6c9a53', 'request_id': 'req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.506427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] The token cache is being invalidate because project cd78b4ee48774a17aadf1b1ecb6c9a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:38.506718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.507425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.509197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1303/2594] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:38 2022] DELETE /identity/v3/projects/cd78b4ee48774a17aadf1b1ecb6c9a53 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:07:38.513464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:38.514815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:38.527928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6195f3e-8a79-49ab-8da2-7e73f2aee35e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:38.586344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:38.586969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:38.587243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:38.587506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] PATH_INFO: `/v3/users/b66d4035812645faadc455185be0a760` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:38.588664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66d4035812645faadc455185be0a760)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:38.590605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:38.640272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66d4035812645faadc455185be0a760'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.648737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66d4035812645faadc455185be0a760'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.649429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66d4035812645faadc455185be0a760'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.656499 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66d4035812645faadc455185be0a760'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.668803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a8be9b2e-a557-424a-955f-6a98025826f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66d4035812645faadc455185be0a760'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:38.670245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:38.675956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1292/2595] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:38 2022] DELETE /identity/v3/users/b66d4035812645faadc455185be0a760 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:38.959468 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:38.960958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:38.975579 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3110-196b-4e0a-b2fc-79b7e2ea0c75 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.042113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.042757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.043035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.043308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] PATH_INFO: `/v3/projects/82d149f12c64447b8b51ec4308c9fca6` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.044462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d149f12c64447b8b51ec4308c9fca6)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.046422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.104047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d149f12c64447b8b51ec4308c9fca6', 'request_id': 'req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:39.115236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d149f12c64447b8b51ec4308c9fca6', 'request_id': 'req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:39.123179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d149f12c64447b8b51ec4308c9fca6', 'request_id': 'req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:39.138537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] The token cache is being invalidate because project 82d149f12c64447b8b51ec4308c9fca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:39.138860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:39.139667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-e3c85e9d-b348-4ded-80e8-73acf6be2f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:39.144743 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1304/2596] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:38 2022] DELETE /identity/v3/projects/82d149f12c64447b8b51ec4308c9fca6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:39.146467 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc6b77-6e27-4b63-8c68-c5d234a02a11 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.146946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc6b77-6e27-4b63-8c68-c5d234a02a11 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.147052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc6b77-6e27-4b63-8c68-c5d234a02a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.176761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-45fc6b77-6e27-4b63-8c68-c5d234a02a11 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:39.178205 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-45fc6b77-6e27-4b63-8c68-c5d234a02a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:39.256415 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1293/2597] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:39.265614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc487d-d1aa-4787-912c-ba0bc7bb9d7b None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.265902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc487d-d1aa-4787-912c-ba0bc7bb9d7b None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.266175 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc487d-d1aa-4787-912c-ba0bc7bb9d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.286338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1fbc487d-d1aa-4787-912c-ba0bc7bb9d7b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:39.287471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1fbc487d-d1aa-4787-912c-ba0bc7bb9d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:39.295379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1305/2598] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:39.299663 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc6b77-6e27-4b63-8c68-c5d234a02a11 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.309561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc6b77-6e27-4b63-8c68-c5d234a02a11 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.314494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.315116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.315386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.315686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.316346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:07:39.316686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.318325 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.334110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1294/2599] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:07:39.335330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc487d-d1aa-4787-912c-ba0bc7bb9d7b None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.341144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:39.345602 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc487d-d1aa-4787-912c-ba0bc7bb9d7b None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.351894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.352616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.353265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.353695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.354555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.356498 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.389232 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1306/2600] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:07:39 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:39.390446 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.400713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e51f6-3abf-4f67-a130-a399c35cfec5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.459212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.459819 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.460103 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.460369 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.461172 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.463129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.486664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1295/2601] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:07:39 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:07:39.490905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.502631 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efcc1a-3c68-4c38-b8c8-fe6b1d9fdea0 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.508654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.509566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.509843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.510110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.510816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.512451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.523222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1307/2602] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:39.527282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.537646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5bf1b-ca37-4c32-abd5-b261323998c4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.542601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d7994-df8a-48eb-b3e5-823ab88ab4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.543233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d7994-df8a-48eb-b3e5-823ab88ab4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.543508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d7994-df8a-48eb-b3e5-823ab88ab4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.543775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d7994-df8a-48eb-b3e5-823ab88ab4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.544468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d7994-df8a-48eb-b3e5-823ab88ab4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.546671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d7994-df8a-48eb-b3e5-823ab88ab4f2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.558545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1296/2603] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:07:39.562959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.573455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab8919-e935-487d-bc77-bc2b7c8523df admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.578323 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.578977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.579254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.579524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] PATH_INFO: `/v3/projects/7f565adb790148108d7cd7e7e310ce2f/users/69a5c62024fc4839b9843a21e8e43e57/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.594790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f565adb790148108d7cd7e7e310ce2f, user_id=69a5c62024fc4839b9843a21e8e43e57, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.597510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.614497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1308/2604] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:07:39 2022] PUT /identity/v3/projects/7f565adb790148108d7cd7e7e310ce2f/users/69a5c62024fc4839b9843a21e8e43e57/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:39.628347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.628758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.629136 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.640335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.642924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:39.659541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] MFA Rules not processed for user `69a5c62024fc4839b9843a21e8e43e57`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:39.660908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:39.724231 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:39.727369 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739cc76-149d-48d2-ab5a-8b29a4e3d7f7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.735957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1297/2605] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:39.741789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.742418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.742698 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.742970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.744753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:39.746948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:39.748366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:39.759113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.760844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.771489 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1309/2606] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:39.805465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9c3d8-b289-4f83-a241-85967fd7d337 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:39.809431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b550e-27b6-4efb-b866-8099f6923a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:39.810049 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b550e-27b6-4efb-b866-8099f6923a7e service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:39.810315 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b550e-27b6-4efb-b866-8099f6923a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:39.810582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b550e-27b6-4efb-b866-8099f6923a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:39.819390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b550e-27b6-4efb-b866-8099f6923a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:39.820899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b550e-27b6-4efb-b866-8099f6923a7e service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:39.829564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1298/2607] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:07:41.781870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:41.783432 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:41.796953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11946c96-ff72-4553-9ade-10ec78c536f3 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:41.802460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:41.803056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:41.803335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:41.803603 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:41.805313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:41.817449 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:41.819029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:41.829202 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1310/2608] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:48.550610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf93f-ceee-4827-8550-43726c150b63 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:48.551080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf93f-ceee-4827-8550-43726c150b63 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:48.551160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf93f-ceee-4827-8550-43726c150b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:48.573007 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-89cdf93f-ceee-4827-8550-43726c150b63 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:48.574245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-89cdf93f-ceee-4827-8550-43726c150b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:48.631811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1299/2609] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 13 18:07:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:48.645015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:48.646444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:48.716854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2cbd9-62c7-4cc4-a24c-b0b8008c05c2 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:48.722241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:48.722863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:48.723137 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:48.723407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:48.737304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:48.738952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:48.748072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1311/2610] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:07:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:07:53.035160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:53.035765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:53.035765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:53.058791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:53.060245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:53.070359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1300/2611] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 13 18:07:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:53.085370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:53.100394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae47e6-dad3-4421-8ded-e7eb5686088a service swift] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:53.107406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:53.108057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:53.108318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:53.108650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:53.119187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:53.122208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:53.124839 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:53.135420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1312/2612] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:07:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 18:07:58.231026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:58.232743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:58.304705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca39d7-f3d9-4738-890d-9eb9f84b3024 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:58.308689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef137e-2583-4269-a16d-8a3ed698a066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:58.309584 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef137e-2583-4269-a16d-8a3ed698a066 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:58.309962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef137e-2583-4269-a16d-8a3ed698a066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:58.310331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef137e-2583-4269-a16d-8a3ed698a066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:58.312487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6bef137e-2583-4269-a16d-8a3ed698a066 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:58.369319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:07:58.398645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef137e-2583-4269-a16d-8a3ed698a066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:58.400677 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef137e-2583-4269-a16d-8a3ed698a066 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:58.411980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1301/2613] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:58.617775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:58.630071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed190e-9f7e-47b4-ba13-b788f7925c12 service swift] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:58.636686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:58.637555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:58.637837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:58.638110 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] PATH_INFO: `/v3/users/69a5c62024fc4839b9843a21e8e43e57` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:58.639277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a5c62024fc4839b9843a21e8e43e57)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:58.641437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:58.708045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a5c62024fc4839b9843a21e8e43e57'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:58.717684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a5c62024fc4839b9843a21e8e43e57'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:58.718680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a5c62024fc4839b9843a21e8e43e57'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:58.726637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a5c62024fc4839b9843a21e8e43e57'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:58.749783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a5c62024fc4839b9843a21e8e43e57'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:58.756888 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1313/2614] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:07:58 2022] DELETE /identity/v3/users/69a5c62024fc4839b9843a21e8e43e57 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:07:58.761795 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:58.762375 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:58.762980 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:58.785755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:07:58.786971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:58.865262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1302/2615] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:07:58.889607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:58.891480 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:58.975221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1275a6a-67b2-4e59-9361-e4d2504cc9a8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:58.979196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:58.979821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:58.980103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:58.980375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:58.992753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:58.995140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:59.012038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1314/2616] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:07:59.723166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:07:59.725625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:07:59.771387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef7d3a-8046-4d91-9a70-8a4ff8f1bd8c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:07:59.776636 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:07:59.777758 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:07:59.778258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:07:59.778811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] PATH_INFO: `/v3/projects/7f565adb790148108d7cd7e7e310ce2f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:07:59.782349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f565adb790148108d7cd7e7e310ce2f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:07:59.785560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:07:59.850154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f565adb790148108d7cd7e7e310ce2f', 'request_id': 'req-0f870f6c-142f-4e44-a682-d665fa016f09'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:59.865357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f565adb790148108d7cd7e7e310ce2f', 'request_id': 'req-0f870f6c-142f-4e44-a682-d665fa016f09'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:59.872118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f565adb790148108d7cd7e7e310ce2f', 'request_id': 'req-0f870f6c-142f-4e44-a682-d665fa016f09'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:59.885973 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] The token cache is being invalidate because project 7f565adb790148108d7cd7e7e310ce2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:07:59.886571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:59.887245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:07:59.888798 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1303/2617] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:07:59 2022] DELETE /identity/v3/projects/7f565adb790148108d7cd7e7e310ce2f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:00.763058 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:00.764724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:00.778486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364aff0-47c6-4b9b-8690-4d79c66c32d5 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:00.846161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:00.847094 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:00.847590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:00.848148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] PATH_INFO: `/v3/users/7880064ff0aa448d9c367c89eba145ff` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:00.849894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7880064ff0aa448d9c367c89eba145ff)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:00.852029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:00.868197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:00.913062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7880064ff0aa448d9c367c89eba145ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:00.924168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7880064ff0aa448d9c367c89eba145ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:00.926871 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7880064ff0aa448d9c367c89eba145ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:00.935820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7880064ff0aa448d9c367c89eba145ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:00.967877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7880064ff0aa448d9c367c89eba145ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:00.975193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1315/2618] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:00 2022] DELETE /identity/v3/users/7880064ff0aa448d9c367c89eba145ff => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:01.174548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:01.176164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:01.191109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f870f6c-142f-4e44-a682-d665fa016f09 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:01.261624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:01.262268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:01.262536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:01.262809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] PATH_INFO: `/v3/projects/2f6832936c824066a4370cc45c5b0282` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:01.263983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6832936c824066a4370cc45c5b0282)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:01.265948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:01.277251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:01.318693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6832936c824066a4370cc45c5b0282', 'request_id': 'req-ebcad689-6de4-410a-bee5-4b642acace13'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.327902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6832936c824066a4370cc45c5b0282', 'request_id': 'req-ebcad689-6de4-410a-bee5-4b642acace13'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.335889 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6832936c824066a4370cc45c5b0282', 'request_id': 'req-ebcad689-6de4-410a-bee5-4b642acace13'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.357522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] The token cache is being invalidate because project 2f6832936c824066a4370cc45c5b0282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:01.357826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.358600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.363648 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1304/2619] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:01 2022] DELETE /identity/v3/projects/2f6832936c824066a4370cc45c5b0282 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:01.364807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:01.366370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:01.381493 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88e5a6-3642-41e7-9b8d-c9de0eb93e70 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:01.449409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:01.450060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:01.450381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:01.450674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] PATH_INFO: `/v3/users/2f05415eeaf24ab5aaafdeb100f48890` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:01.451829 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f05415eeaf24ab5aaafdeb100f48890)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:01.453961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:01.514153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f05415eeaf24ab5aaafdeb100f48890'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.523339 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f05415eeaf24ab5aaafdeb100f48890'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.523995 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f05415eeaf24ab5aaafdeb100f48890'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.531482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f05415eeaf24ab5aaafdeb100f48890'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.546189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f05415eeaf24ab5aaafdeb100f48890'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.555037 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1316/2620] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:01 2022] DELETE /identity/v3/users/2f05415eeaf24ab5aaafdeb100f48890 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:08:01.730957 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:01.732364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:01.770373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcad689-6de4-410a-bee5-4b642acace13 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:01.844084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:01.844881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:01.845116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:01.845653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] PATH_INFO: `/v3/projects/21add0dce4a24d13816aa9987bc7af81` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:01.852094 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21add0dce4a24d13816aa9987bc7af81)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:01.854649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:01.920831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21add0dce4a24d13816aa9987bc7af81', 'request_id': 'req-86bc8ca6-f77d-4357-890e-5cdee4dcea86'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.936510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21add0dce4a24d13816aa9987bc7af81', 'request_id': 'req-86bc8ca6-f77d-4357-890e-5cdee4dcea86'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.943678 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21add0dce4a24d13816aa9987bc7af81', 'request_id': 'req-86bc8ca6-f77d-4357-890e-5cdee4dcea86'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.957658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] The token cache is being invalidate because project 21add0dce4a24d13816aa9987bc7af81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:01.957842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.960731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:01.962558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1305/2621] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:01 2022] DELETE /identity/v3/projects/21add0dce4a24d13816aa9987bc7af81 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:02.353573 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:02.355028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:02.370194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff1508-73e3-4f83-b580-a41a920b6615 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:02.375208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:02.429525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:02.430161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:02.430440 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:02.430771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] PATH_INFO: `/v3/users/0625b3113ce044bbac4f6142eaade11c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:02.431922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0625b3113ce044bbac4f6142eaade11c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:02.433941 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:02.487686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0625b3113ce044bbac4f6142eaade11c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.495313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0625b3113ce044bbac4f6142eaade11c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.496064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0625b3113ce044bbac4f6142eaade11c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.503670 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0625b3113ce044bbac4f6142eaade11c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.515884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-714e7119-80d4-437d-9e7e-f236763ff3b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0625b3113ce044bbac4f6142eaade11c'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.521900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1317/2622] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:02 2022] DELETE /identity/v3/users/0625b3113ce044bbac4f6142eaade11c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:02.694017 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:02.695567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:02.711465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc8ca6-f77d-4357-890e-5cdee4dcea86 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:02.752474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:02.789640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:02.790271 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:02.790549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:02.790833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] PATH_INFO: `/v3/projects/a279fcbf274c4704a31a2b95061e1469` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:02.792155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a279fcbf274c4704a31a2b95061e1469)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:02.794111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:02.847066 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a279fcbf274c4704a31a2b95061e1469', 'request_id': 'req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.855485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a279fcbf274c4704a31a2b95061e1469', 'request_id': 'req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.861676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a279fcbf274c4704a31a2b95061e1469', 'request_id': 'req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.872267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] The token cache is being invalidate because project a279fcbf274c4704a31a2b95061e1469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:02.872580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.873254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6cab5bbf-357f-4b21-a0e4-5659f0e3acb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:02.874853 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1306/2623] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:02 2022] DELETE /identity/v3/projects/a279fcbf274c4704a31a2b95061e1469 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:02.880528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed5a8-2a98-4f4a-8e9e-17a6d26089bf None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:02.880808 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed5a8-2a98-4f4a-8e9e-17a6d26089bf None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:02.881084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ed5a8-2a98-4f4a-8e9e-17a6d26089bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:02.901689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ab6ed5a8-2a98-4f4a-8e9e-17a6d26089bf None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:02.902902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ab6ed5a8-2a98-4f4a-8e9e-17a6d26089bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:02.967276 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1318/2624] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:02.977379 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61dd7-5539-4d41-b78d-3b83e256ad7d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:02.977664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61dd7-5539-4d41-b78d-3b83e256ad7d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:02.977940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf61dd7-5539-4d41-b78d-3b83e256ad7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:02.999789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cbf61dd7-5539-4d41-b78d-3b83e256ad7d None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:03.001062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cbf61dd7-5539-4d41-b78d-3b83e256ad7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:03.010258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1307/2625] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:03.015792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ed5a8-2a98-4f4a-8e9e-17a6d26089bf None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:03.025923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ed5a8-2a98-4f4a-8e9e-17a6d26089bf None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:03.031350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:03.031950 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.032243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.032564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.033226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:08:03.033526 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:03.035141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:03.046993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1319/2626] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:08:03.051359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf61dd7-5539-4d41-b78d-3b83e256ad7d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:03.061349 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf61dd7-5539-4d41-b78d-3b83e256ad7d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:03.066448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:03.067070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.067323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.067591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.068229 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:03.070193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:03.105055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1308/2627] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:03 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:03.110553 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:03.121371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c1c59-3ace-4ed9-a788-772a40ed1032 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:03.136124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:03.180874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:03.181474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.181757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.182025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.182658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:03.184601 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:03.208197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1320/2628] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:03 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:03.212601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:03.223510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fdb9d-6e38-4b34-9418-7efdd798979b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:03.228784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:03.229377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.229649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.229918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.230550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:03.232154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:03.244474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1309/2629] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:03.249342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:03.265065 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530c10f-2aea-4be6-837b-896741e0d407 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:03.270936 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aa537-290e-48c3-8205-d8a85847b22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:03.271540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa537-290e-48c3-8205-d8a85847b22c admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.271821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa537-290e-48c3-8205-d8a85847b22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.272069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aa537-290e-48c3-8205-d8a85847b22c admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.272760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aa537-290e-48c3-8205-d8a85847b22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:03.274377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aa537-290e-48c3-8205-d8a85847b22c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:03.285492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1321/2630] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:03.289641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:03.299898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274775f1-aabe-4b74-86be-d656ea90de47 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:03.305029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:03.305647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.305908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.306201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] PATH_INFO: `/v3/projects/f871818268dd4a66b12581d11d592dd9/users/4b90387188ed414989c13ec3e0f130ca/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.321075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f871818268dd4a66b12581d11d592dd9, user_id=4b90387188ed414989c13ec3e0f130ca, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:03.323733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:03.338320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1310/2631] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:03 2022] PUT /identity/v3/projects/f871818268dd4a66b12581d11d592dd9/users/4b90387188ed414989c13ec3e0f130ca/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:03.343682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf52e6-6ab0-4819-b749-fa5291466403 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.343973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf52e6-6ab0-4819-b749-fa5291466403 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.344233 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf52e6-6ab0-4819-b749-fa5291466403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.363939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8faf52e6-6ab0-4819-b749-fa5291466403 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:03.365205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8faf52e6-6ab0-4819-b749-fa5291466403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:03.427224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1322/2632] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:03.437480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:03.439656 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:03.512756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c24a1-61ee-4e53-bf1c-2c8ded825e3f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:03.517910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:03.518421 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:03.518733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:03.519000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:03.529505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:03.531069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:03.534933 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:03.542693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1311/2633] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:04.183727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f332-197d-41a5-9b0a-67b1acfce89e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:04.184017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f332-197d-41a5-9b0a-67b1acfce89e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:04.184287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f332-197d-41a5-9b0a-67b1acfce89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:04.213625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-11a9f332-197d-41a5-9b0a-67b1acfce89e None None] MFA Rules not processed for user `4b90387188ed414989c13ec3e0f130ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:04.214960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-11a9f332-197d-41a5-9b0a-67b1acfce89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:04.286639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1323/2634] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:04 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:04.291423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:04.301931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb786d-e54a-4dc5-a034-a47197277884 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:04.306900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:04.307502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:04.307778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:04.308046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:04.308706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:04.310652 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:04.339626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1312/2635] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:04 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:04.344022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f332-197d-41a5-9b0a-67b1acfce89e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:04.354804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f332-197d-41a5-9b0a-67b1acfce89e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:04.374809 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:04.545907 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:04.546525 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:04.546799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:04.547062 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:04.547704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:04.549640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:04.573900 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1324/2636] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:04 2022] POST /identity/v3/users => generated 309 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:08:04.578213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:04.588796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca01b50-7a69-4726-a414-a8be052a0cac admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:04.594010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:04.594617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:04.594888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:04.595153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:04.595842 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:04.597490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:04.608276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1313/2637] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:04.612561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:04.623033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e3ec0-4142-45f8-b305-8b6e21e4dc12 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:04.629334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:04.629980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:04.630273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:04.630532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] PATH_INFO: `/v3/projects/427dd13647384c47b7cd2eb2a3c3e76a/users/7ba9d654b1e64118a03ba5f439ac10ff/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:04.645905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] RBAC: Authorizing `identity:create_grant(project_id=427dd13647384c47b7cd2eb2a3c3e76a, user_id=7ba9d654b1e64118a03ba5f439ac10ff, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:04.648552 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:04.662401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1325/2638] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:04 2022] PUT /identity/v3/projects/427dd13647384c47b7cd2eb2a3c3e76a/users/7ba9d654b1e64118a03ba5f439ac10ff/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:04.666363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:04.676516 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb97bbb-5645-4f3e-80dd-e2b978ea3af4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:04.738575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083b895-2f4f-4490-ad72-66c1ad8331d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:04.739318 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083b895-2f4f-4490-ad72-66c1ad8331d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:04.739585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083b895-2f4f-4490-ad72-66c1ad8331d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:04.739845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083b895-2f4f-4490-ad72-66c1ad8331d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:04.740520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083b895-2f4f-4490-ad72-66c1ad8331d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:04.742141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083b895-2f4f-4490-ad72-66c1ad8331d4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:04.753182 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1314/2639] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:04.757650 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:04.768623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0400c9d-0625-4691-8036-94c7f78b489c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:04.773610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:04.774236 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:04.774509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:04.774776 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] PATH_INFO: `/v3/projects/427dd13647384c47b7cd2eb2a3c3e76a/users/7ba9d654b1e64118a03ba5f439ac10ff/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:04.776764 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] RBAC: Authorizing `identity:create_grant(project_id=427dd13647384c47b7cd2eb2a3c3e76a, user_id=7ba9d654b1e64118a03ba5f439ac10ff, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:04.779395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:04.789730 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1326/2640] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:04 2022] PUT /identity/v3/projects/427dd13647384c47b7cd2eb2a3c3e76a/users/7ba9d654b1e64118a03ba5f439ac10ff/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:05.442575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85580ac2-552d-4312-82b7-a85e7fe5027a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:05.443137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85580ac2-552d-4312-82b7-a85e7fe5027a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:05.443137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85580ac2-552d-4312-82b7-a85e7fe5027a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:05.472590 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-85580ac2-552d-4312-82b7-a85e7fe5027a None None] MFA Rules not processed for user `7ba9d654b1e64118a03ba5f439ac10ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:05.473805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-85580ac2-552d-4312-82b7-a85e7fe5027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:05.546206 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1315/2641] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:06.610935 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:06.675301 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad3722-aa86-4d90-96e9-9939894b266a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:06.680332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:06.681029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:06.681307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:06.681574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] PATH_INFO: `/v3/users/4b90387188ed414989c13ec3e0f130ca` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:06.682774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b90387188ed414989c13ec3e0f130ca)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:06.684721 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:06.736269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b90387188ed414989c13ec3e0f130ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:06.746022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b90387188ed414989c13ec3e0f130ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:06.746736 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b90387188ed414989c13ec3e0f130ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:06.754510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b90387188ed414989c13ec3e0f130ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:06.765990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:06.767773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b90387188ed414989c13ec3e0f130ca'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:06.774580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1327/2642] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:06 2022] DELETE /identity/v3/users/4b90387188ed414989c13ec3e0f130ca => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:06.963256 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85580ac2-552d-4312-82b7-a85e7fe5027a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:06.976617 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85580ac2-552d-4312-82b7-a85e7fe5027a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:07.022288 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:07.035744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:07.036396 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.036696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.036960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] PATH_INFO: `/v3/projects/f871818268dd4a66b12581d11d592dd9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.038078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f871818268dd4a66b12581d11d592dd9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:07.039983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:07.086428 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f871818268dd4a66b12581d11d592dd9', 'request_id': 'req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.095024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f871818268dd4a66b12581d11d592dd9', 'request_id': 'req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.101755 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f871818268dd4a66b12581d11d592dd9', 'request_id': 'req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.116772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] The token cache is being invalidate because project f871818268dd4a66b12581d11d592dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:07.117068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.117865 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.119580 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1316/2643] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:06 2022] DELETE /identity/v3/projects/f871818268dd4a66b12581d11d592dd9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:07.123799 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:07.125377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:07.139025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926b4e-a68b-46f5-b13a-9546e0570385 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:07.196982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:07.197610 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.197882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.198154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] PATH_INFO: `/v3/users/7ba9d654b1e64118a03ba5f439ac10ff` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.199381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba9d654b1e64118a03ba5f439ac10ff)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:07.201336 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:07.254972 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba9d654b1e64118a03ba5f439ac10ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.263387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba9d654b1e64118a03ba5f439ac10ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.264151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba9d654b1e64118a03ba5f439ac10ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.271464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba9d654b1e64118a03ba5f439ac10ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.282968 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-4501118d-6fd6-46ca-940b-02b2bce08967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba9d654b1e64118a03ba5f439ac10ff'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.289223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1328/2644] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:07 2022] DELETE /identity/v3/users/7ba9d654b1e64118a03ba5f439ac10ff => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:07.478279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:07.479794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:07.493169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa243e3-65da-4f7f-845e-f3d7cbc21d37 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:07.553570 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:07.554164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.554560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.554818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] PATH_INFO: `/v3/projects/427dd13647384c47b7cd2eb2a3c3e76a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.556114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=427dd13647384c47b7cd2eb2a3c3e76a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:07.558095 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:07.606106 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427dd13647384c47b7cd2eb2a3c3e76a', 'request_id': 'req-aefeeb15-927b-4915-875d-14f2e2864f2f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.613319 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427dd13647384c47b7cd2eb2a3c3e76a', 'request_id': 'req-aefeeb15-927b-4915-875d-14f2e2864f2f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.619536 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427dd13647384c47b7cd2eb2a3c3e76a', 'request_id': 'req-aefeeb15-927b-4915-875d-14f2e2864f2f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.630488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] The token cache is being invalidate because project 427dd13647384c47b7cd2eb2a3c3e76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:07.630776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.631406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-aefeeb15-927b-4915-875d-14f2e2864f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:07.632967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1317/2645] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:07 2022] DELETE /identity/v3/projects/427dd13647384c47b7cd2eb2a3c3e76a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:07.637945 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09286d9-2572-444e-9e2f-524e59e45375 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.638411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09286d9-2572-444e-9e2f-524e59e45375 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.638505 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09286d9-2572-444e-9e2f-524e59e45375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.659331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-c09286d9-2572-444e-9e2f-524e59e45375 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:07.660578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c09286d9-2572-444e-9e2f-524e59e45375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:07.722294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1329/2646] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:07.732083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8fd9-d15e-4ae7-aeaf-2aeea5082c79 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.732347 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8fd9-d15e-4ae7-aeaf-2aeea5082c79 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.732731 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8fd9-d15e-4ae7-aeaf-2aeea5082c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.753861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:07.760409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-451a8fd9-d15e-4ae7-aeaf-2aeea5082c79 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:07.762389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-451a8fd9-d15e-4ae7-aeaf-2aeea5082c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:07.771579 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1318/2647] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:07.775987 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09286d9-2572-444e-9e2f-524e59e45375 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:07.785981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09286d9-2572-444e-9e2f-524e59e45375 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:07.791014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:07.791623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.791898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.792173 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.792862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:08:07.793167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:07.794805 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:07.806600 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1330/2648] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:08:07.810909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a8fd9-d15e-4ae7-aeaf-2aeea5082c79 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:07.820435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a8fd9-d15e-4ae7-aeaf-2aeea5082c79 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:07.825473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:07.826060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.826336 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.826604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.827242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:07.829316 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:07.862425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1319/2649] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:07 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:07.866728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:07.871946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:07.878706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3510ba-f26c-41bd-b80d-c3ee515609eb admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:07.947707 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:07.948555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:07.948902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:07.949246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:07.950200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:07.953360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:07.977000 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1331/2650] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:07 2022] POST /identity/v3/users => generated 279 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:07.982834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:07.993901 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d11ae4-9d84-4a46-b2a9-9e6eca69205e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:07.999691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.000294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.000613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.000891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.001530 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.003519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.013976 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1320/2651] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:08.018196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.028597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea0e60-bea1-4424-8325-25277518042d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.033710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464272-20b5-4d43-b0fe-fc37f5c9e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.034299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464272-20b5-4d43-b0fe-fc37f5c9e5da admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.034576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464272-20b5-4d43-b0fe-fc37f5c9e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.034846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464272-20b5-4d43-b0fe-fc37f5c9e5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.035506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464272-20b5-4d43-b0fe-fc37f5c9e5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.037194 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464272-20b5-4d43-b0fe-fc37f5c9e5da admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.047135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1332/2652] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:08.051246 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.061569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4755e6a-3d82-45d8-8236-67a548df1d26 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.066727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.067365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.067638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.067905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] PATH_INFO: `/v3/projects/0e89717a358f439e8f31764ac22a0fff/users/d701dfd2fa3e4790aaaa17231e8aa953/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.082809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e89717a358f439e8f31764ac22a0fff, user_id=d701dfd2fa3e4790aaaa17231e8aa953, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.085549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.101263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1321/2653] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:08 2022] PUT /identity/v3/projects/0e89717a358f439e8f31764ac22a0fff/users/d701dfd2fa3e4790aaaa17231e8aa953/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:08:08.112203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c77fe-df0b-4b3d-b611-728223c3cb03 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.112533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c77fe-df0b-4b3d-b611-728223c3cb03 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.112813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c77fe-df0b-4b3d-b611-728223c3cb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.143024 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-613c77fe-df0b-4b3d-b611-728223c3cb03 None None] MFA Rules not processed for user `d701dfd2fa3e4790aaaa17231e8aa953`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:08.144241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-613c77fe-df0b-4b3d-b611-728223c3cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:08.207896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1333/2654] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:08.212660 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.277639 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66772c-b91c-4090-a772-e70207f3ccb7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.282900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.283497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.283851 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.284120 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.284775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.286682 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.315861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1322/2655] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:08 2022] POST /identity/v3/projects => generated 355 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:08.320025 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c77fe-df0b-4b3d-b611-728223c3cb03 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.330532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c77fe-df0b-4b3d-b611-728223c3cb03 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.390042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.390607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.390898 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.391275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.392101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.394376 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.412060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:08.422964 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1334/2656] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:08 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:08.424174 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.434609 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85a705-3d2b-47ca-8b5d-be8e85b4a869 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.439690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.440457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.440764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.441037 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.441704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.443469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.454983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1323/2657] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:08:08.458927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.468929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66d336-cc01-449e-943f-39359c6ef2cc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.474144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.474786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.475061 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.475330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] PATH_INFO: `/v3/projects/9ef17c219d6442e89aff0dff511bbb19/users/e6ad50aa91dc45c38eab419d5709daa0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.490431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef17c219d6442e89aff0dff511bbb19, user_id=e6ad50aa91dc45c38eab419d5709daa0, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.493101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.511564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1335/2658] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:08 2022] PUT /identity/v3/projects/9ef17c219d6442e89aff0dff511bbb19/users/e6ad50aa91dc45c38eab419d5709daa0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:08.515642 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.525811 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2be9b6-1532-4408-96c3-e9d572b6453e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.582105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d433a-30c5-4fba-a946-555e1f85bc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.582626 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d433a-30c5-4fba-a946-555e1f85bc4d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.582898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d433a-30c5-4fba-a946-555e1f85bc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.583160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d433a-30c5-4fba-a946-555e1f85bc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.583794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d433a-30c5-4fba-a946-555e1f85bc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.585430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d433a-30c5-4fba-a946-555e1f85bc4d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.594765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1324/2659] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:08.599649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.615413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5535b-95a9-4e15-b287-65c8ac967ba9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.622853 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.623830 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.624274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.624717 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] PATH_INFO: `/v3/projects/9ef17c219d6442e89aff0dff511bbb19/users/e6ad50aa91dc45c38eab419d5709daa0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.627997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef17c219d6442e89aff0dff511bbb19, user_id=e6ad50aa91dc45c38eab419d5709daa0, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.630633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.641540 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1336/2660] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:08 2022] PUT /identity/v3/projects/9ef17c219d6442e89aff0dff511bbb19/users/e6ad50aa91dc45c38eab419d5709daa0/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:08:08.652552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e047c26-aca0-4dcf-bbd7-bb48cacee1fc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.652837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e047c26-aca0-4dcf-bbd7-bb48cacee1fc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.653195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e047c26-aca0-4dcf-bbd7-bb48cacee1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.679623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:08.682807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7e047c26-aca0-4dcf-bbd7-bb48cacee1fc None None] MFA Rules not processed for user `e6ad50aa91dc45c38eab419d5709daa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:08.685027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7e047c26-aca0-4dcf-bbd7-bb48cacee1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:08.757368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1325/2661] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:08:08.769237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.771302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:08.828195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee1e42-34d4-4265-93ee-53fbbd1bd84c admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.832748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.833273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.833562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.833833 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:08.843340 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:08.844790 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:08.854644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1337/2662] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:08:08.934410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e047c26-aca0-4dcf-bbd7-bb48cacee1fc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:08.944692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e047c26-aca0-4dcf-bbd7-bb48cacee1fc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:08.948524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759559d-83f7-40c1-9023-8823df35591f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:08.949125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759559d-83f7-40c1-9023-8823df35591f service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:08.949400 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759559d-83f7-40c1-9023-8823df35591f service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:08.949664 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759559d-83f7-40c1-9023-8823df35591f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:09.012691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759559d-83f7-40c1-9023-8823df35591f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:09.014155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759559d-83f7-40c1-9023-8823df35591f service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:09.023405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1326/2663] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:18.849921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=41,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:19.019356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=56 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:24.154992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:24.218532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732cc91-fb5b-482e-98da-974b6b3f0d22 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:24.224814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:24.225444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:24.225741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:24.226046 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] PATH_INFO: `/v3/users/d701dfd2fa3e4790aaaa17231e8aa953` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:24.227425 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d701dfd2fa3e4790aaaa17231e8aa953)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:24.229422 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:24.280910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd701dfd2fa3e4790aaaa17231e8aa953'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.289920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd701dfd2fa3e4790aaaa17231e8aa953'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.290565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd701dfd2fa3e4790aaaa17231e8aa953'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.300109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd701dfd2fa3e4790aaaa17231e8aa953'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.314511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd701dfd2fa3e4790aaaa17231e8aa953'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.321045 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1338/2664] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:24 2022] DELETE /identity/v3/users/d701dfd2fa3e4790aaaa17231e8aa953 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:24.326625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:24.327148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:24.327259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:24.350587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:24.351776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:24.412576 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1327/2665] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:24.422592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:24.424107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:24.482692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca478e-fff1-449e-96b5-cfd1d632329f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:24.487272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:24.487879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:24.488148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:24.488453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:24.498285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:24.499715 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:24.510118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1339/2666] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:08:24.795359 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:24.796944 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:24.810750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163622d-9b84-497a-8cc0-f37b94689c8a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:24.816928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:24.817675 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:24.818060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:24.818353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] PATH_INFO: `/v3/projects/0e89717a358f439e8f31764ac22a0fff` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:24.819730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e89717a358f439e8f31764ac22a0fff)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:24.821969 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:24.875916 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e89717a358f439e8f31764ac22a0fff', 'request_id': 'req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.884377 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e89717a358f439e8f31764ac22a0fff', 'request_id': 'req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.890849 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e89717a358f439e8f31764ac22a0fff', 'request_id': 'req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.902257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] The token cache is being invalidate because project 0e89717a358f439e8f31764ac22a0fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:24.902548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.903197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:24.904750 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1328/2667] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:24 2022] DELETE /identity/v3/projects/0e89717a358f439e8f31764ac22a0fff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:24.908969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:24.918564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf6f45-f9a9-4cba-a784-519919a9b1d5 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:24.936851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:24.976951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:24.977655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:24.977932 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:24.978197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] PATH_INFO: `/v3/users/e6ad50aa91dc45c38eab419d5709daa0` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:24.979360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ad50aa91dc45c38eab419d5709daa0)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:24.981307 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.030188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ad50aa91dc45c38eab419d5709daa0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.038719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ad50aa91dc45c38eab419d5709daa0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.039353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ad50aa91dc45c38eab419d5709daa0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.047892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ad50aa91dc45c38eab419d5709daa0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.059915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-3e1f1a3e-d675-4573-9ea3-f97b9ef9f15f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ad50aa91dc45c38eab419d5709daa0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.065952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1340/2668] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:24 2022] DELETE /identity/v3/users/e6ad50aa91dc45c38eab419d5709daa0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:08:25.346622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.348133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:25.362723 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc00eb-3ad3-4e8b-adb2-8f5f29c67053 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:25.421333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:25.421891 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.422160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.422425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] PATH_INFO: `/v3/projects/9ef17c219d6442e89aff0dff511bbb19` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.423538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef17c219d6442e89aff0dff511bbb19)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:25.425490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.456528 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:25.473293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef17c219d6442e89aff0dff511bbb19', 'request_id': 'req-45c83ad7-99cd-4e0c-9322-f1b672427d76'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.481586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef17c219d6442e89aff0dff511bbb19', 'request_id': 'req-45c83ad7-99cd-4e0c-9322-f1b672427d76'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.487939 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef17c219d6442e89aff0dff511bbb19', 'request_id': 'req-45c83ad7-99cd-4e0c-9322-f1b672427d76'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.499330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] The token cache is being invalidate because project 9ef17c219d6442e89aff0dff511bbb19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:25.499624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.500378 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-45c83ad7-99cd-4e0c-9322-f1b672427d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:25.502450 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1329/2669] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:25 2022] DELETE /identity/v3/projects/9ef17c219d6442e89aff0dff511bbb19 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:25.507734 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740216f-8fdf-42e4-b4e5-608c16497866 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.508223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740216f-8fdf-42e4-b4e5-608c16497866 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.508322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740216f-8fdf-42e4-b4e5-608c16497866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.528090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-9740216f-8fdf-42e4-b4e5-608c16497866 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:25.529292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9740216f-8fdf-42e4-b4e5-608c16497866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:25.596328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1341/2670] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:08:25.606983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bcdc7-e8c0-47b5-8997-cd854bdf74f1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.607272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bcdc7-e8c0-47b5-8997-cd854bdf74f1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.607545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bcdc7-e8c0-47b5-8997-cd854bdf74f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.628401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9e0bcdc7-e8c0-47b5-8997-cd854bdf74f1 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:25.629647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9e0bcdc7-e8c0-47b5-8997-cd854bdf74f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:25.638598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1330/2671] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:25.643266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740216f-8fdf-42e4-b4e5-608c16497866 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.653519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740216f-8fdf-42e4-b4e5-608c16497866 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:25.659224 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:25.659824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.660099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.660369 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.661052 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:08:25.661372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:25.662961 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.675040 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1342/2672] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:08:25.679468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bcdc7-e8c0-47b5-8997-cd854bdf74f1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.689282 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bcdc7-e8c0-47b5-8997-cd854bdf74f1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:25.695305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:25.695903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.696176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.696565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.697183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:25.699080 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.733185 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1331/2673] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:25 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:25.734456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.747719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab511a58-84d1-40d5-8e5c-f0389867705f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:25.774430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:25.807460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:25.808056 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.808333 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.808624 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.809243 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:25.811143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.834423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1343/2674] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:25 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:25.838788 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.849166 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8155f3b-3270-41ce-903b-a8894be7eb7f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:25.853900 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:25.854488 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.854745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.855014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.855653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:25.857373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.867286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1332/2675] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:25.871315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.882283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cd6a0-455a-4b17-b7ad-ebd3855a6137 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:25.887144 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a9e4-7d35-49ca-ae26-75771d398cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:25.887738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a9e4-7d35-49ca-ae26-75771d398cab admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.888043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a9e4-7d35-49ca-ae26-75771d398cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.888345 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a9e4-7d35-49ca-ae26-75771d398cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.889055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a9e4-7d35-49ca-ae26-75771d398cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:25.890643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a9e4-7d35-49ca-ae26-75771d398cab admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.901929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1344/2676] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:25.906158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.917646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0babb5-cc6a-4c3d-bc55-aec9ac651207 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:25.922624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:25.923281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.923562 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.923890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] PATH_INFO: `/v3/projects/a8ca2a4607d3423ebb414ad5a6b55a48/users/cd4f8ea06ee34d3e92a77efbbde9fb21/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.939365 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ca2a4607d3423ebb414ad5a6b55a48, user_id=cd4f8ea06ee34d3e92a77efbbde9fb21, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:25.942057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:25.958647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1333/2677] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:25 2022] PUT /identity/v3/projects/a8ca2a4607d3423ebb414ad5a6b55a48/users/cd4f8ea06ee34d3e92a77efbbde9fb21/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:25.970984 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f52c4-3e29-41f1-b7f7-226d5a881ac1 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:25.971360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f52c4-3e29-41f1-b7f7-226d5a881ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:25.971674 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f52c4-3e29-41f1-b7f7-226d5a881ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:25.996402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:25.998154 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:26.004506 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-751f52c4-3e29-41f1-b7f7-226d5a881ac1 None None] MFA Rules not processed for user `cd4f8ea06ee34d3e92a77efbbde9fb21`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:26.005669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-751f52c4-3e29-41f1-b7f7-226d5a881ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:26.056995 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2ea7f-a6da-48d1-97ef-5ebec1a4813f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.060806 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.061414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.061692 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.062011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.063742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:26.070608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1345/2678] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:26.075057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f52c4-3e29-41f1-b7f7-226d5a881ac1 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.139815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f52c4-3e29-41f1-b7f7-226d5a881ac1 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.144686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.145502 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.145845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.146042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.146677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.148646 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.179840 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1346/2679] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:26 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:08:26.183922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.209561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:26.247463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f6b53-c8d8-4a7b-ae20-6c025eea0f41 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.252789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.253401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.253686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.253953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.254591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.256576 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.258262 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.259780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.265141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:26.272470 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1334/2680] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:08:26.280454 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1347/2681] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:26 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:08:26.284337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.294448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b4601-fd88-4b42-a66c-10b12f12f105 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.299560 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.300210 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.300594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.300863 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.301502 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.303096 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.313372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1335/2682] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:08:26.317587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.328298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b815ef7-1bc2-4730-bee2-5ed7e21b7ad8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.334804 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.337107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.337107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.337107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] PATH_INFO: `/v3/projects/bd219222d2a0497baff8a15125eb692c/users/ac78ab5ce9c8431da769261ff2e35163/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.353029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd219222d2a0497baff8a15125eb692c, user_id=ac78ab5ce9c8431da769261ff2e35163, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.355785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.370529 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1348/2683] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:26 2022] PUT /identity/v3/projects/bd219222d2a0497baff8a15125eb692c/users/ac78ab5ce9c8431da769261ff2e35163/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:08:26.374589 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.385294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0d897-cdbf-4cff-bfd6-276c0b219c11 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.444072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976b3c8-b1b7-4846-968f-b451bfe8d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.444720 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976b3c8-b1b7-4846-968f-b451bfe8d1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.444994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976b3c8-b1b7-4846-968f-b451bfe8d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.445257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976b3c8-b1b7-4846-968f-b451bfe8d1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.446014 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976b3c8-b1b7-4846-968f-b451bfe8d1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.447747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976b3c8-b1b7-4846-968f-b451bfe8d1d2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.457821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1336/2684] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:26.461755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.472357 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cff0d-1830-4b4a-b200-a56ea580aec8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.477015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.477708 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.477984 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.478256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] PATH_INFO: `/v3/projects/bd219222d2a0497baff8a15125eb692c/users/ac78ab5ce9c8431da769261ff2e35163/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.480287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd219222d2a0497baff8a15125eb692c, user_id=ac78ab5ce9c8431da769261ff2e35163, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.482912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.493930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1349/2685] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:26 2022] PUT /identity/v3/projects/bd219222d2a0497baff8a15125eb692c/users/ac78ab5ce9c8431da769261ff2e35163/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:26.505602 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ace4e-df70-4a2b-b4e7-d7dce0d83fb2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.505886 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ace4e-df70-4a2b-b4e7-d7dce0d83fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.506168 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ace4e-df70-4a2b-b4e7-d7dce0d83fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.533273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-664ace4e-df70-4a2b-b4e7-d7dce0d83fb2 None None] MFA Rules not processed for user `ac78ab5ce9c8431da769261ff2e35163`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:26.534478 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-664ace4e-df70-4a2b-b4e7-d7dce0d83fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:26.602133 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1337/2686] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:26.613406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.615267 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:26.675120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27b52b-ad7d-4a76-b385-09f13c821664 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.680249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49504e4-3d73-43a2-baed-54a3cd55e660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.681005 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49504e4-3d73-43a2-baed-54a3cd55e660 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.681298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49504e4-3d73-43a2-baed-54a3cd55e660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.681623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49504e4-3d73-43a2-baed-54a3cd55e660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.692014 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49504e4-3d73-43a2-baed-54a3cd55e660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.693512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49504e4-3d73-43a2-baed-54a3cd55e660 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.703211 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1350/2687] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:26.734188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ace4e-df70-4a2b-b4e7-d7dce0d83fb2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.796442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ace4e-df70-4a2b-b4e7-d7dce0d83fb2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:26.801116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:26.801793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.802067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.802331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] PATH_INFO: `/v3/users/cd4f8ea06ee34d3e92a77efbbde9fb21` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.803458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4f8ea06ee34d3e92a77efbbde9fb21)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:26.805413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:26.818512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:26.854686 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4f8ea06ee34d3e92a77efbbde9fb21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:26.862718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4f8ea06ee34d3e92a77efbbde9fb21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:26.864013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4f8ea06ee34d3e92a77efbbde9fb21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:26.871986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4f8ea06ee34d3e92a77efbbde9fb21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:26.885705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4f8ea06ee34d3e92a77efbbde9fb21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:26.891807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1338/2688] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:26 2022] DELETE /identity/v3/users/cd4f8ea06ee34d3e92a77efbbde9fb21 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:26.896405 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:26.896720 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:26.896994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:26.915850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:26.916889 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:26.977133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1351/2689] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:26.987432 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:26.989024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:27.048145 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d47b8-a0f7-410e-873a-1a00ca6dd2d3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:27.051885 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:27.052527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:27.052804 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:27.053075 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:27.062717 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:27.064238 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:27.073981 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1339/2690] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:27.377472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:27.379104 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:27.392026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb39803-df74-469f-b6ee-dfd70b57f8d7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:27.397559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:27.398167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:27.398443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:27.398710 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] PATH_INFO: `/v3/projects/a8ca2a4607d3423ebb414ad5a6b55a48` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:27.399904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ca2a4607d3423ebb414ad5a6b55a48)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:27.401854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:27.415229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:27.453209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ca2a4607d3423ebb414ad5a6b55a48', 'request_id': 'req-0de6beaf-4f5d-44ed-a909-4c27652ef99a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.460916 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ca2a4607d3423ebb414ad5a6b55a48', 'request_id': 'req-0de6beaf-4f5d-44ed-a909-4c27652ef99a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.467858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ca2a4607d3423ebb414ad5a6b55a48', 'request_id': 'req-0de6beaf-4f5d-44ed-a909-4c27652ef99a'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.479998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] The token cache is being invalidate because project a8ca2a4607d3423ebb414ad5a6b55a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:27.480195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.480899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.482435 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1352/2691] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:27 2022] DELETE /identity/v3/projects/a8ca2a4607d3423ebb414ad5a6b55a48 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:27.486635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:27.497515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1d90d-3550-4489-9a10-204a614a9db9 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:27.556062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:27.556807 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:27.557083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:27.557356 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] PATH_INFO: `/v3/users/ac78ab5ce9c8431da769261ff2e35163` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:27.558523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac78ab5ce9c8431da769261ff2e35163)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:27.560462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:27.603861 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:27.614559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac78ab5ce9c8431da769261ff2e35163'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.622761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac78ab5ce9c8431da769261ff2e35163'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.623597 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac78ab5ce9c8431da769261ff2e35163'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.631487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac78ab5ce9c8431da769261ff2e35163'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.643688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9cbb3f6b-8e1d-46a9-9d44-81bf70f5cb62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac78ab5ce9c8431da769261ff2e35163'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:27.649868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1340/2692] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:27 2022] DELETE /identity/v3/users/ac78ab5ce9c8431da769261ff2e35163 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:28.058034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.059897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:28.073797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6beaf-4f5d-44ed-a909-4c27652ef99a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.140134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.140798 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.141073 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.141344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] PATH_INFO: `/v3/projects/bd219222d2a0497baff8a15125eb692c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.142472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd219222d2a0497baff8a15125eb692c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.144522 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.224346 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd219222d2a0497baff8a15125eb692c', 'request_id': 'req-c907a45c-62c9-4164-b280-192ad1114e58'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:28.239314 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd219222d2a0497baff8a15125eb692c', 'request_id': 'req-c907a45c-62c9-4164-b280-192ad1114e58'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:28.249494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd219222d2a0497baff8a15125eb692c', 'request_id': 'req-c907a45c-62c9-4164-b280-192ad1114e58'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:28.269208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] The token cache is being invalidate because project bd219222d2a0497baff8a15125eb692c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:28.269937 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:28.272049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c907a45c-62c9-4164-b280-192ad1114e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:28.279009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1353/2693] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:28 2022] DELETE /identity/v3/projects/bd219222d2a0497baff8a15125eb692c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:28.280770 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b3b1-2e88-4c34-83b2-146166a742ff None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.282146 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b3b1-2e88-4c34-83b2-146166a742ff None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.282440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b3b1-2e88-4c34-83b2-146166a742ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.304630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4aa5b3b1-2e88-4c34-83b2-146166a742ff None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:28.305966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4aa5b3b1-2e88-4c34-83b2-146166a742ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:28.408742 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1341/2694] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:28.416851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca4442-eebc-40d7-84c9-17dbaa52ca66 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.417160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca4442-eebc-40d7-84c9-17dbaa52ca66 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.417564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca4442-eebc-40d7-84c9-17dbaa52ca66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.439453 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e7ca4442-eebc-40d7-84c9-17dbaa52ca66 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:28.440835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e7ca4442-eebc-40d7-84c9-17dbaa52ca66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:28.449760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1354/2695] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:28.455740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5b3b1-2e88-4c34-83b2-146166a742ff None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.467625 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5b3b1-2e88-4c34-83b2-146166a742ff None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.473151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.473888 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.474083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.474307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.475307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:08:28.475450 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.477162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.494274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1342/2696] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:08:28.495645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca4442-eebc-40d7-84c9-17dbaa52ca66 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.507196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca4442-eebc-40d7-84c9-17dbaa52ca66 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.513856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.514208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.514297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.514524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.517146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.517146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.545738 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1355/2697] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:28 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:28.549968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.559771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d06108-de89-4d15-b997-0ff21f02a533 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.615785 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.616397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.616702 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.616971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.617601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.621076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.645393 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1343/2698] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:28 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:28.649852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.655318 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:28.663067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6797-b4a2-443f-9ba0-d3a3b75d412b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.669577 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.670297 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.670578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.670848 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.671491 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.673120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.684781 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1356/2699] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:08:28.689124 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.699358 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b0f89-f030-4754-bf6a-848f1e6a7c6c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.703985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec705cf-ace3-41c6-8769-d8207953f5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.704613 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec705cf-ace3-41c6-8769-d8207953f5ee admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.704897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec705cf-ace3-41c6-8769-d8207953f5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.705164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec705cf-ace3-41c6-8769-d8207953f5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.705816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec705cf-ace3-41c6-8769-d8207953f5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.707423 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec705cf-ace3-41c6-8769-d8207953f5ee admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.718988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1344/2700] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:28.723467 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.734017 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4f1a8-0f98-49ee-b0b7-61d696cb33d1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.738970 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.739649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.739926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.740203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] PATH_INFO: `/v3/projects/76c760106f2146be964755698d8ebd9f/users/47448346c75842458a6e47dc52cd917f/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.757296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c760106f2146be964755698d8ebd9f, user_id=47448346c75842458a6e47dc52cd917f, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.759982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.774332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1357/2701] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:28 2022] PUT /identity/v3/projects/76c760106f2146be964755698d8ebd9f/users/47448346c75842458a6e47dc52cd917f/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:28.784727 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.785034 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.785310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.821113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] MFA Rules not processed for user `47448346c75842458a6e47dc52cd917f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:28.822389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:28.829984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:28.903492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1345/2702] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:28.915204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:28.917360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:28.975353 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce9492-cd94-4c43-aa52-083db8945a2a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:28.979029 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:28.979645 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:28.979922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:28.980189 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:28.988850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:28.990290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:28.999330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1358/2703] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:29.741657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:29.743118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:29.808330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbc9b8-1689-4b29-a5ff-fa44d6425981 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:29.813051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:29.813709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:29.813968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:29.814227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] PATH_INFO: `/v3/users/e68ecd9b3ca643539b109f75bb242d6f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:29.815408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68ecd9b3ca643539b109f75bb242d6f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:29.817351 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:29.863978 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68ecd9b3ca643539b109f75bb242d6f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:29.872691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68ecd9b3ca643539b109f75bb242d6f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:29.873362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68ecd9b3ca643539b109f75bb242d6f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:29.881141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68ecd9b3ca643539b109f75bb242d6f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:29.894745 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5e3d989a-9605-4455-8072-40e6a7884afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68ecd9b3ca643539b109f75bb242d6f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:29.901127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1346/2704] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:29 2022] DELETE /identity/v3/users/e68ecd9b3ca643539b109f75bb242d6f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:30.101105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.114657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c63e06-51e2-4907-a5a0-4db4ceed876a service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.173330 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.173977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.174258 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.174526 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] PATH_INFO: `/v3/projects/75e14fd88bfb4cd896fa7b070a0fac47` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.175660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e14fd88bfb4cd896fa7b070a0fac47)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.177615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.228395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e14fd88bfb4cd896fa7b070a0fac47', 'request_id': 'req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:30.237193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e14fd88bfb4cd896fa7b070a0fac47', 'request_id': 'req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:30.245259 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e14fd88bfb4cd896fa7b070a0fac47', 'request_id': 'req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:30.253640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:30.258047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] The token cache is being invalidate because project 75e14fd88bfb4cd896fa7b070a0fac47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:30.258800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:30.259678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-b6c826de-09cc-45d1-9a8f-a49c3ee990fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:30.261737 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1359/2705] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:30 2022] DELETE /identity/v3/projects/75e14fd88bfb4cd896fa7b070a0fac47 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:30.266599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995e14-22f6-46ac-a03f-d3fe010db956 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.266965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995e14-22f6-46ac-a03f-d3fe010db956 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.267299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e995e14-22f6-46ac-a03f-d3fe010db956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.290386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2e995e14-22f6-46ac-a03f-d3fe010db956 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:30.291606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e995e14-22f6-46ac-a03f-d3fe010db956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:30.313045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:30.354543 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1347/2706] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:30.363121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d24e41-d39d-4280-913a-c52f503893e3 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.363431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d24e41-d39d-4280-913a-c52f503893e3 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.363703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d24e41-d39d-4280-913a-c52f503893e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.384274 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-e6d24e41-d39d-4280-913a-c52f503893e3 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:30.385558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e6d24e41-d39d-4280-913a-c52f503893e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:30.397038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1360/2707] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:30.398473 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e995e14-22f6-46ac-a03f-d3fe010db956 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.408751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e995e14-22f6-46ac-a03f-d3fe010db956 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.416548 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.417366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.417646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.417911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.418959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:08:30.418959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.420501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.436476 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1348/2708] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:08:30.438982 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d24e41-d39d-4280-913a-c52f503893e3 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.448835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d24e41-d39d-4280-913a-c52f503893e3 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.454269 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.454998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.455337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.455667 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.456362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.458260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.486974 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1361/2709] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:30 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:30.491389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.503390 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec070-e8ee-4b30-acce-b84418be7949 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.567845 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.568619 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.569035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.569397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.570308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.573244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.596455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1349/2710] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:30 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:08:30.599512 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.609896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d330-c1b9-4b90-a194-65de1a5502e2 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.614479 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.615077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.615348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.615614 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.616251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.618012 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.629085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1362/2711] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:30.633164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.643404 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27cb5e-943f-4435-b319-6c93ce435dfc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.648273 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51dc3-70e2-4173-bf80-013cce0e74c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.648895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51dc3-70e2-4173-bf80-013cce0e74c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.649171 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51dc3-70e2-4173-bf80-013cce0e74c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.649469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51dc3-70e2-4173-bf80-013cce0e74c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.650117 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd51dc3-70e2-4173-bf80-013cce0e74c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.651730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd51dc3-70e2-4173-bf80-013cce0e74c6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.662441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1350/2712] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:30.666727 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.676813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe9fc3-4418-4595-80d4-172b92ae95c4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.681969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.682633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.682899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.683159 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] PATH_INFO: `/v3/projects/629311eda0584a5fbb7d8995cf27540c/users/714cd09ba19b492f9a8c80343abcf6b9/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.698992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629311eda0584a5fbb7d8995cf27540c, user_id=714cd09ba19b492f9a8c80343abcf6b9, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.701678 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.714874 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1363/2713] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:30 2022] PUT /identity/v3/projects/629311eda0584a5fbb7d8995cf27540c/users/714cd09ba19b492f9a8c80343abcf6b9/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:30.720195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c627823-8e2e-47fe-9934-df54f6b5b024 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.720577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c627823-8e2e-47fe-9934-df54f6b5b024 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.720847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c627823-8e2e-47fe-9934-df54f6b5b024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.740829 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0c627823-8e2e-47fe-9934-df54f6b5b024 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:30.742035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0c627823-8e2e-47fe-9934-df54f6b5b024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:30.804160 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1351/2714] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:30.813940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:30.815938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:30.874013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373944c7-8420-4b7d-a3e1-8967bc0a35a9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:30.878528 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:30.879197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:30.879406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:30.879681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:30.889078 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:30.890500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:30.899796 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1364/2715] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:34.878592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2b9d0-b2d9-4327-afd3-4db40171ca62 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:34.879155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2b9d0-b2d9-4327-afd3-4db40171ca62 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:34.879155 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2b9d0-b2d9-4327-afd3-4db40171ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:34.925552 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3fb2b9d0-b2d9-4327-afd3-4db40171ca62 None None] MFA Rules not processed for user `714cd09ba19b492f9a8c80343abcf6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:34.939821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3fb2b9d0-b2d9-4327-afd3-4db40171ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:34.961647 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:35.014263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1352/2716] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:35.029567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:35.031156 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:35.107789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8930d08-379e-4c3e-a695-59db8d66fc95 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:35.113228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:35.113846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:35.114121 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:35.114394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:35.123064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:35.124714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:35.134680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1365/2717] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:36.448989 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203434dd-e883-4b8d-9046-6bc56315c57e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:36.449558 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203434dd-e883-4b8d-9046-6bc56315c57e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:36.449673 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203434dd-e883-4b8d-9046-6bc56315c57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:36.468503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-203434dd-e883-4b8d-9046-6bc56315c57e None None] MFA Rules not processed for user `714cd09ba19b492f9a8c80343abcf6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:36.469701 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-203434dd-e883-4b8d-9046-6bc56315c57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:36.476340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1353/2718] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:40.724726 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:40.735096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba5aa7-01fd-4df2-b1c8-824b10972868 service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:40.740197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:40.740844 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:40.741163 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:40.741429 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:40.743626 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:40.767423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:40.806411 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:40.808001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:40.817536 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1366/2719] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:46.463211 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=20 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:47.013691 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203434dd-e883-4b8d-9046-6bc56315c57e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:47.023671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203434dd-e883-4b8d-9046-6bc56315c57e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:47.027546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7207b4-2e02-407e-ab5d-22ce94cbb629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:47.028221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7207b4-2e02-407e-ab5d-22ce94cbb629 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:47.028545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7207b4-2e02-407e-ab5d-22ce94cbb629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:47.028817 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7207b4-2e02-407e-ab5d-22ce94cbb629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:47.030620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b7207b4-2e02-407e-ab5d-22ce94cbb629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:47.043332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7207b4-2e02-407e-ab5d-22ce94cbb629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:47.044955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7207b4-2e02-407e-ab5d-22ce94cbb629 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:47.055533 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1354/2720] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:47.137296 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:47.138753 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:47.151873 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c663-e499-4121-b65d-c347bb89c35c service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:47.156583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:47.157303 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:47.157654 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:47.157977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] PATH_INFO: `/v3/users/47448346c75842458a6e47dc52cd917f` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:47.159321 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47448346c75842458a6e47dc52cd917f)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:47.161360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:47.210226 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47448346c75842458a6e47dc52cd917f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.220099 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47448346c75842458a6e47dc52cd917f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.220814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47448346c75842458a6e47dc52cd917f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.228532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47448346c75842458a6e47dc52cd917f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.244816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47448346c75842458a6e47dc52cd917f'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.253575 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1367/2721] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:47 2022] DELETE /identity/v3/users/47448346c75842458a6e47dc52cd917f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:47.258254 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:47.258618 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:47.258962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:47.284233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:47.286237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:47.350515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1355/2722] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:08:47.360416 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:47.362092 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:47.425375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1a7ca-ef19-4898-acb3-ec6301d9da85 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:47.429398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7f59e-4e49-41cd-a631-6b345c388c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:47.430013 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7f59e-4e49-41cd-a631-6b345c388c20 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:47.430406 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7f59e-4e49-41cd-a631-6b345c388c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:47.430682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7f59e-4e49-41cd-a631-6b345c388c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:47.439852 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7f59e-4e49-41cd-a631-6b345c388c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:47.441451 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7f59e-4e49-41cd-a631-6b345c388c20 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:47.451268 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1368/2723] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:47.721766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:47.724443 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:47.746429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c946d5-6d7e-44c0-8a7b-32ed0b5f0997 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:47.759477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:47.760275 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:47.760601 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:47.760868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] PATH_INFO: `/v3/projects/76c760106f2146be964755698d8ebd9f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:47.762024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c760106f2146be964755698d8ebd9f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:47.763968 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:47.818958 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c760106f2146be964755698d8ebd9f', 'request_id': 'req-3ae4a693-158a-4adc-b963-2c7208a1a497'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.827352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c760106f2146be964755698d8ebd9f', 'request_id': 'req-3ae4a693-158a-4adc-b963-2c7208a1a497'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.835372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c760106f2146be964755698d8ebd9f', 'request_id': 'req-3ae4a693-158a-4adc-b963-2c7208a1a497'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.857931 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] The token cache is being invalidate because project 76c760106f2146be964755698d8ebd9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:47.858500 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.859684 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae4a693-158a-4adc-b963-2c7208a1a497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:47.864911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1356/2724] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:47 2022] DELETE /identity/v3/projects/76c760106f2146be964755698d8ebd9f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:47.872349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d340b-edab-4d2f-b742-fe89628c4347 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:47.872349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d340b-edab-4d2f-b742-fe89628c4347 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:47.872349 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d340b-edab-4d2f-b742-fe89628c4347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:47.892027 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-cb5d340b-edab-4d2f-b742-fe89628c4347 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:47.893282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-cb5d340b-edab-4d2f-b742-fe89628c4347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:47.928877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:47.973709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1369/2725] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:47.975394 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee8d9b-6040-4ae6-b636-d3ef5930668c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:47.975683 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee8d9b-6040-4ae6-b636-d3ef5930668c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:47.975928 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee8d9b-6040-4ae6-b636-d3ef5930668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.003539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c6ee8d9b-6040-4ae6-b636-d3ef5930668c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:48.004808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c6ee8d9b-6040-4ae6-b636-d3ef5930668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:48.015716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1357/2726] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:08:48.019756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d340b-edab-4d2f-b742-fe89628c4347 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:48.030863 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d340b-edab-4d2f-b742-fe89628c4347 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:48.036130 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:48.036595 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.036948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.037195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.038039 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:08:48.038354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:48.040760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:48.077143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1370/2727] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:08:48.078461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee8d9b-6040-4ae6-b636-d3ef5930668c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:48.091215 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee8d9b-6040-4ae6-b636-d3ef5930668c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:48.097097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:48.097856 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.098307 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.098733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.099700 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:48.103398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:48.225010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1358/2728] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:08:48 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:48.230420 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:48.245754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16bc85-0abf-46d1-a272-35548001222f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:48.329071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:48.329697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.329977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.330248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.331585 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:48.333607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:48.360485 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1371/2729] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:08:48 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:08:48.364834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:48.378457 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36c1dd-f7b3-4010-9991-16de9983681a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:48.383455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:48.384067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.384388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.384699 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.385431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:48.387055 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:48.403464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1359/2730] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:08:48.404821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:48.416511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3288d7-f8e2-49da-a47e-8bf095e1cd4b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:48.422127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9fb6c-c9c7-433a-8dfc-689beb7ba709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:48.422976 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9fb6c-c9c7-433a-8dfc-689beb7ba709 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.423441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9fb6c-c9c7-433a-8dfc-689beb7ba709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.423891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9fb6c-c9c7-433a-8dfc-689beb7ba709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.425412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9fb6c-c9c7-433a-8dfc-689beb7ba709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:48.427725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9fb6c-c9c7-433a-8dfc-689beb7ba709 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:48.441927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1372/2731] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:08:48.446346 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:48.457204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d229beb-4715-461e-aa01-738512d6bdf2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:48.462073 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:48.462808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.463085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.463448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] PATH_INFO: `/v3/projects/695556c06be64e648553c00aae591f89/users/1926ee1dc3834f0dbd5580d42f3919a5/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.478444 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695556c06be64e648553c00aae591f89, user_id=1926ee1dc3834f0dbd5580d42f3919a5, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:48.481193 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:48.495412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1360/2732] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:08:48 2022] PUT /identity/v3/projects/695556c06be64e648553c00aae591f89/users/1926ee1dc3834f0dbd5580d42f3919a5/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:48.515204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.515571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.515910 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.548709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] MFA Rules not processed for user `1926ee1dc3834f0dbd5580d42f3919a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:48.550048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:48.621589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1373/2733] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:48.633266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:48.635308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:48.646093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:48.700796 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdfbb4-ffd7-4dc3-935b-2a4e786d9c97 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:48.704974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:48.705600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:48.705876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:48.706149 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:48.715862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:48.717372 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:48.727214 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1361/2734] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:50.674254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:50.676055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:50.752246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:50.809604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8a7a4-69ca-410b-8aa7-633dbd1e24f9 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:50.819239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:50.819239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:50.819239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:50.819239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:50.819975 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:50.837689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:50.839756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:50.851148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1374/2735] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:54.931355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:54.944846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc8f98-05c5-47ef-b1d1-5c98e2df6556 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:54.951266 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:54.952086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:54.952581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:54.953088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:54.955158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:54.968201 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:54.969986 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:54.984567 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1362/2736] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:55.006613 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:55.008124 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:55.093709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11f65b-c947-42e9-b375-411d76220fb9 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:55.098151 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:55.098786 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:55.099087 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:55.099383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] PATH_INFO: `/v3/users/2ab9930d6ce54cacbb3d4fd6313294d5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:55.100587 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab9930d6ce54cacbb3d4fd6313294d5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:55.102497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:55.153375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab9930d6ce54cacbb3d4fd6313294d5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.163053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab9930d6ce54cacbb3d4fd6313294d5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.163846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab9930d6ce54cacbb3d4fd6313294d5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.171379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab9930d6ce54cacbb3d4fd6313294d5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.185265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab9930d6ce54cacbb3d4fd6313294d5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.198352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1375/2737] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:55 2022] DELETE /identity/v3/users/2ab9930d6ce54cacbb3d4fd6313294d5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:55.383294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:55.397031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78d786-a4b5-4df9-a5f9-d4388d1873cd service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:55.464761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:55.465401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:55.465710 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:55.465984 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] PATH_INFO: `/v3/projects/5eafd54351bd4049b87ad308a3638f80` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:55.467138 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eafd54351bd4049b87ad308a3638f80)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:55.469112 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:55.519053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eafd54351bd4049b87ad308a3638f80', 'request_id': 'req-fed577ae-0593-4691-a528-7316da2f7716'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.531344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eafd54351bd4049b87ad308a3638f80', 'request_id': 'req-fed577ae-0593-4691-a528-7316da2f7716'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.538028 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eafd54351bd4049b87ad308a3638f80', 'request_id': 'req-fed577ae-0593-4691-a528-7316da2f7716'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.549789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] The token cache is being invalidate because project 5eafd54351bd4049b87ad308a3638f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:55.550195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.550894 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.552564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1363/2738] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:55 2022] DELETE /identity/v3/projects/5eafd54351bd4049b87ad308a3638f80 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:55.556580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:55.558034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:55.571100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f99bdf-a634-44f1-9b88-4bd8d60a222f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:55.611238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:55.638998 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:55.639762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:55.639926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:55.640195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] PATH_INFO: `/v3/users/6f3e442d656f4e6eb1eaf367e4c70238` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:55.641343 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3e442d656f4e6eb1eaf367e4c70238)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:55.643235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:55.691920 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3e442d656f4e6eb1eaf367e4c70238'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.701793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3e442d656f4e6eb1eaf367e4c70238'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.702823 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3e442d656f4e6eb1eaf367e4c70238'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.710957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3e442d656f4e6eb1eaf367e4c70238'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.725101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3e442d656f4e6eb1eaf367e4c70238'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:55.731503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1376/2739] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:55 2022] DELETE /identity/v3/users/6f3e442d656f4e6eb1eaf367e4c70238 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:55.909391 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:55.910992 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:55.925919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed577ae-0593-4691-a528-7316da2f7716 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:55.952412 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:55.986634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:55.987323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:55.987594 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:55.987862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] PATH_INFO: `/v3/projects/1e4aa136d4384ac7a9bcb795533eec73` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:55.989033 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4aa136d4384ac7a9bcb795533eec73)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:55.990952 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:56.037492 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4aa136d4384ac7a9bcb795533eec73', 'request_id': 'req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:56.045592 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4aa136d4384ac7a9bcb795533eec73', 'request_id': 'req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:56.052373 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4aa136d4384ac7a9bcb795533eec73', 'request_id': 'req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:56.063072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] The token cache is being invalidate because project 1e4aa136d4384ac7a9bcb795533eec73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:56.063353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:56.063974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cfe60a22-d1c7-4d7a-9dca-dd38df88cab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:56.065465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1364/2740] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:55 2022] DELETE /identity/v3/projects/1e4aa136d4384ac7a9bcb795533eec73 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:59.081417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:59.083101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:59.096766 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c67aab-a618-4a4d-9f70-542fa352372e admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:59.157070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:59.157699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:59.157981 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:59.158252 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] PATH_INFO: `/v3/users/1926ee1dc3834f0dbd5580d42f3919a5` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:59.159419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1926ee1dc3834f0dbd5580d42f3919a5)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:59.161382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:59.211921 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1926ee1dc3834f0dbd5580d42f3919a5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.219160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1926ee1dc3834f0dbd5580d42f3919a5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.220011 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1926ee1dc3834f0dbd5580d42f3919a5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.227026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1926ee1dc3834f0dbd5580d42f3919a5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.228100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:59.242264 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1926ee1dc3834f0dbd5580d42f3919a5'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.248788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1377/2741] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:08:59 2022] DELETE /identity/v3/users/1926ee1dc3834f0dbd5580d42f3919a5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:59.253854 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:59.254353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:59.254353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:59.274243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:59.275458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:59.343239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1365/2742] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:59.353379 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:59.355022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:59.415283 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37b750-5dde-41ec-be12-2727bed8c901 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:59.419407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a8056-662d-49d7-9685-c1bff07dd37f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:59.420001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a8056-662d-49d7-9685-c1bff07dd37f service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:59.420281 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a8056-662d-49d7-9685-c1bff07dd37f service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:59.420583 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a8056-662d-49d7-9685-c1bff07dd37f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:59.431824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a8056-662d-49d7-9685-c1bff07dd37f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:59.433706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a8056-662d-49d7-9685-c1bff07dd37f service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:59.443469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1378/2743] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:08:59.735805 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:08:59.737308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:59.749219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e83bf4-db32-4aae-a5b0-e67f79347d43 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:08:59.755847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:08:59.756721 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:59.757027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:59.757371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] PATH_INFO: `/v3/projects/695556c06be64e648553c00aae591f89` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:59.758761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] RBAC: Authorizing `identity:delete_project(project_id=695556c06be64e648553c00aae591f89)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:08:59.760786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:08:59.809328 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695556c06be64e648553c00aae591f89', 'request_id': 'req-b157e776-6f86-419e-87f7-9c37eda4f68c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.811847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:08:59.818152 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695556c06be64e648553c00aae591f89', 'request_id': 'req-b157e776-6f86-419e-87f7-9c37eda4f68c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.826118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695556c06be64e648553c00aae591f89', 'request_id': 'req-b157e776-6f86-419e-87f7-9c37eda4f68c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.836838 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] The token cache is being invalidate because project 695556c06be64e648553c00aae591f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:08:59.837129 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.837837 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b157e776-6f86-419e-87f7-9c37eda4f68c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:08:59.839333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1366/2744] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:08:59 2022] DELETE /identity/v3/projects/695556c06be64e648553c00aae591f89 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:08:59.844510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40811c3b-1c07-480f-871b-d2ebf952ed14 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:59.844747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40811c3b-1c07-480f-871b-d2ebf952ed14 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:59.845004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40811c3b-1c07-480f-871b-d2ebf952ed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:59.865382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-40811c3b-1c07-480f-871b-d2ebf952ed14 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:59.866588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-40811c3b-1c07-480f-871b-d2ebf952ed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:59.925608 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1379/2745] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:59.935640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeabcdb-5cdc-4484-8cfc-95cc426b0505 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:08:59.936032 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeabcdb-5cdc-4484-8cfc-95cc426b0505 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:08:59.936380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeabcdb-5cdc-4484-8cfc-95cc426b0505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:08:59.974257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-faeabcdb-5cdc-4484-8cfc-95cc426b0505 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:08:59.975561 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-faeabcdb-5cdc-4484-8cfc-95cc426b0505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:08:59.985966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1367/2746] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:08:59.990615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40811c3b-1c07-480f-871b-d2ebf952ed14 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:00.001856 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40811c3b-1c07-480f-871b-d2ebf952ed14 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:00.008033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:00.008676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.008971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.009246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.009905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:09:00.010220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:00.011944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:00.024192 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1380/2747] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:09:00.030541 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeabcdb-5cdc-4484-8cfc-95cc426b0505 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:00.042577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeabcdb-5cdc-4484-8cfc-95cc426b0505 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:00.047585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:00.048141 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.048413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.048714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.049338 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:00.051299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:00.080360 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1368/2748] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:09:00 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:09:00.084625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:00.094688 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac406a-d7d9-453d-b3ed-dfb2e304f7dc admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:00.151431 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:00.152105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.152388 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.152694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.153341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:00.155263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:00.174771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:09:00.181069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1381/2749] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:09:00 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:09:00.184826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:00.195081 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f1286-86e2-4b7d-8a2a-948cffb05cc1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:00.200105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:00.200724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.200994 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.201258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.201890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:00.203489 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:00.213550 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1369/2750] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:09:00.217487 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:00.228220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349995a-dbe2-405e-8fea-1c0177e1c782 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:00.233697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9937f0c-4ca4-47a8-9462-54ac0a24f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:00.234338 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9937f0c-4ca4-47a8-9462-54ac0a24f431 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.234615 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9937f0c-4ca4-47a8-9462-54ac0a24f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.234891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9937f0c-4ca4-47a8-9462-54ac0a24f431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.235524 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9937f0c-4ca4-47a8-9462-54ac0a24f431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:00.237169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9937f0c-4ca4-47a8-9462-54ac0a24f431 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:00.249701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1382/2751] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:09:00.254409 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:00.264778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e4b29-e41f-4198-bcfd-86e8699819b5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:00.270469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:00.271162 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.271439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.271680 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] PATH_INFO: `/v3/projects/1271ab6e6c9b4c93b05aafddbdbf9afe/users/f379ce3f38fc41568493be7493e75736/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.290363 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1271ab6e6c9b4c93b05aafddbdbf9afe, user_id=f379ce3f38fc41568493be7493e75736, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:00.293109 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:00.307910 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1370/2752] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:09:00 2022] PUT /identity/v3/projects/1271ab6e6c9b4c93b05aafddbdbf9afe/users/f379ce3f38fc41568493be7493e75736/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:09:00.314251 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d260f03-5a6a-48c0-9a03-cae7764cf8dd None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.314562 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d260f03-5a6a-48c0-9a03-cae7764cf8dd None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.314835 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d260f03-5a6a-48c0-9a03-cae7764cf8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.339148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7d260f03-5a6a-48c0-9a03-cae7764cf8dd None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:09:00.340385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7d260f03-5a6a-48c0-9a03-cae7764cf8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:09:00.400020 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1383/2753] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:09:00.413440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:00.416108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:09:00.481764 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80a5fe-c1a5-44e4-96ce-cfb88eb3ee16 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:00.486063 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:00.486658 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:00.486930 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:00.487186 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:00.497102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:00.498595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:00.509180 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1371/2754] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:09:01.143282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9324f9-281c-4827-9507-966ff44df15e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:01.143795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9324f9-281c-4827-9507-966ff44df15e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:01.143906 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9324f9-281c-4827-9507-966ff44df15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:01.171550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-7c9324f9-281c-4827-9507-966ff44df15e None None] MFA Rules not processed for user `f379ce3f38fc41568493be7493e75736`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:09:01.172847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7c9324f9-281c-4827-9507-966ff44df15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:09:01.240179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1384/2755] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:09:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:09:01.251586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:01.253483 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:09:01.312320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5666-4f71-46a3-9675-8b782e41e34d service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:01.316218 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a0b23-c8e6-42b3-b630-9fbabd1ec79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:01.316903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a0b23-c8e6-42b3-b630-9fbabd1ec79e service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:01.317196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a0b23-c8e6-42b3-b630-9fbabd1ec79e service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:01.317465 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a0b23-c8e6-42b3-b630-9fbabd1ec79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:01.326966 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a0b23-c8e6-42b3-b630-9fbabd1ec79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:01.328417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a0b23-c8e6-42b3-b630-9fbabd1ec79e service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:01.337296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1372/2756] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:09:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:09:02.715883 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b829c-0a6a-47a6-b892-2c7e68f7dfbe None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:02.716460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b829c-0a6a-47a6-b892-2c7e68f7dfbe None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:02.716460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b829c-0a6a-47a6-b892-2c7e68f7dfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:02.736780 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-475b829c-0a6a-47a6-b892-2c7e68f7dfbe None None] MFA Rules not processed for user `f379ce3f38fc41568493be7493e75736`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:09:02.738008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-475b829c-0a6a-47a6-b892-2c7e68f7dfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:09:02.746383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1385/2757] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:09:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:09:11.333577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=3,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:09:12.732695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=67 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:09:44.335280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec68d33-8b8f-4f45-acaf-8490793e27eb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:44.335876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec68d33-8b8f-4f45-acaf-8490793e27eb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:44.335876 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec68d33-8b8f-4f45-acaf-8490793e27eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:44.365384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cec68d33-8b8f-4f45-acaf-8490793e27eb None None] MFA Rules not processed for user `714cd09ba19b492f9a8c80343abcf6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:09:44.366599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cec68d33-8b8f-4f45-acaf-8490793e27eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:09:44.441738 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1373/2758] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:09:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:09:44.775718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b829c-0a6a-47a6-b892-2c7e68f7dfbe None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:09:44.796275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b829c-0a6a-47a6-b892-2c7e68f7dfbe None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:09:44.799953 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:09:44.800687 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:09:44.800943 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:09:44.801208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:09:44.802896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:09:44.823905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:09:44.825436 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:09:44.834565 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1386/2759] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:09:54.440145 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=37 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:09:54.830193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=15 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:10:01.972949 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd2300-cbb2-408d-8afe-e6ed8d5c6c7e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:10:01.973510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd2300-cbb2-408d-8afe-e6ed8d5c6c7e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:10:01.973510 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd2300-cbb2-408d-8afe-e6ed8d5c6c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:10:01.995239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-70cd2300-cbb2-408d-8afe-e6ed8d5c6c7e None None] MFA Rules not processed for user `f379ce3f38fc41568493be7493e75736`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:10:01.996905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-70cd2300-cbb2-408d-8afe-e6ed8d5c6c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:10:02.005844 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1374/2760] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:10:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:10:11.988906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:10:34.847329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:10:34.848881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:10:34.949464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a491f7-e4d2-4551-845a-ff137c0c7e99 service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:10:34.955139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:10:34.955757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:10:34.956042 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:10:34.956313 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:10:34.958098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:10:34.970597 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:10:34.972207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:10:34.982623 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1387/2761] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:10:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:10:44.978244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=44 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:10:53.452381 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44960-7642-4cbe-97ac-67967847a117 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:10:53.452961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44960-7642-4cbe-97ac-67967847a117 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:10:53.452961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44960-7642-4cbe-97ac-67967847a117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:10:53.472792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f7d44960-7642-4cbe-97ac-67967847a117 None None] MFA Rules not processed for user `714cd09ba19b492f9a8c80343abcf6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:10:53.473955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f7d44960-7642-4cbe-97ac-67967847a117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:10:53.481501 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1375/2762] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:10:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:10:58.248666 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:10:58.278909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2e83-b9cb-4732-b5fd-9aeeaf8d175b service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:10:58.284923 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:10:58.285588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:10:58.285887 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:10:58.286176 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] PATH_INFO: `/v3/users/f379ce3f38fc41568493be7493e75736` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:10:58.287335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f379ce3f38fc41568493be7493e75736)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:10:58.289305 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:10:58.339009 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f379ce3f38fc41568493be7493e75736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.347980 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f379ce3f38fc41568493be7493e75736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.348810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f379ce3f38fc41568493be7493e75736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.356757 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f379ce3f38fc41568493be7493e75736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.369172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f379ce3f38fc41568493be7493e75736'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.376235 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1388/2763] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:10:58 2022] DELETE /identity/v3/users/f379ce3f38fc41568493be7493e75736 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:10:58.550323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44960-7642-4cbe-97ac-67967847a117 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:10:58.562653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44960-7642-4cbe-97ac-67967847a117 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:10:58.619649 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:10:58.620212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:10:58.620599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:10:58.620878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] PATH_INFO: `/v3/projects/1271ab6e6c9b4c93b05aafddbdbf9afe` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:10:58.622062 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1271ab6e6c9b4c93b05aafddbdbf9afe)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:10:58.623983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:10:58.678299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1271ab6e6c9b4c93b05aafddbdbf9afe', 'request_id': 'req-191d7baa-65a1-42ba-8e9d-62f45f88ec59'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.686955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1271ab6e6c9b4c93b05aafddbdbf9afe', 'request_id': 'req-191d7baa-65a1-42ba-8e9d-62f45f88ec59'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.693689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1271ab6e6c9b4c93b05aafddbdbf9afe', 'request_id': 'req-191d7baa-65a1-42ba-8e9d-62f45f88ec59'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.716585 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] The token cache is being invalidate because project 1271ab6e6c9b4c93b05aafddbdbf9afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:10:58.716870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.717545 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-191d7baa-65a1-42ba-8e9d-62f45f88ec59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:10:58.719079 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1376/2764] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:10:58 2022] DELETE /identity/v3/projects/1271ab6e6c9b4c93b05aafddbdbf9afe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:11:08.371751 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:11:08.714285 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:11:48.344396 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:11:48.345886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:11:48.427550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58ee6f-db7e-4698-bba2-ac427155e84d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:11:48.476686 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:11:48.477293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:11:48.477535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:11:48.477814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:11:48.479744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:11:48.583949 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:11:48.585423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:11:48.606619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1389/2765] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:11:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:11:55.494897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:11:55.495480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:11:55.495890 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:11:55.518791 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] MFA Rules not processed for user `714cd09ba19b492f9a8c80343abcf6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:11:55.520027 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:11:55.583195 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1377/2766] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:11:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:11:55.624096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:11:55.625926 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:11:55.640414 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=100 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:11:55.705958 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddded32-6a86-4e6e-9f8c-a48854a70109 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:11:55.709700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c282d6f237542e3b873100748759794', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:11:55.710438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:11:55.710722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:11:55.710993 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:11:55.712947 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:11:55.727374 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:11:55.728881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:11:55.740408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1390/2767] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:11:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:12:05.575902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=25 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:12:05.582482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:12:05.583955 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:12:05.668077 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9045f6b-41e4-4d15-81f2-887237b3b79c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:12:05.672506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:12:05.673125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:12:05.673401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:12:05.673671 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:12:05.675508 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:12:05.687221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:12:05.688737 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:12:05.697099 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1378/2768] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:12:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 18:12:05.735095 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=30 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:12:15.693604 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=35 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:12:51.118919 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:12:51.120814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:12:51.145371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21998dbb-75ec-4322-9ae1-694a615e73a1 service glance] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:12:51.151514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:12:51.152978 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:12:51.153260 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:12:51.153537 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:12:51.155622 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:12:51.168795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:12:51.170355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:12:51.184748 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1391/2769] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:12:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:12:59.712276 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:12:59.713847 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:12:59.783150 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e8f2a-48c0-4f57-92ba-a78242efb604 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:12:59.787705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:12:59.788333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:12:59.788627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:12:59.788913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] PATH_INFO: `/v3/users/714cd09ba19b492f9a8c80343abcf6b9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:12:59.790044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714cd09ba19b492f9a8c80343abcf6b9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:12:59.791970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:12:59.854253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714cd09ba19b492f9a8c80343abcf6b9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:12:59.863219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714cd09ba19b492f9a8c80343abcf6b9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:12:59.864089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714cd09ba19b492f9a8c80343abcf6b9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:12:59.872599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714cd09ba19b492f9a8c80343abcf6b9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:12:59.885542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-46af7a1c-78ce-41f9-bb32-cbb63e622481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714cd09ba19b492f9a8c80343abcf6b9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:12:59.891630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1379/2770] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:12:59 2022] DELETE /identity/v3/users/714cd09ba19b492f9a8c80343abcf6b9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:00.073695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.085413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dff11-ec7e-4147-9397-8424c2c2deec service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.145225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.145860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.146135 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.146437 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] PATH_INFO: `/v3/projects/629311eda0584a5fbb7d8995cf27540c` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.147684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629311eda0584a5fbb7d8995cf27540c)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.149657 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.205747 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629311eda0584a5fbb7d8995cf27540c', 'request_id': 'req-81567951-c7be-49c4-8d2c-7cf09e33cec1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:00.215419 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629311eda0584a5fbb7d8995cf27540c', 'request_id': 'req-81567951-c7be-49c4-8d2c-7cf09e33cec1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:00.221344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629311eda0584a5fbb7d8995cf27540c', 'request_id': 'req-81567951-c7be-49c4-8d2c-7cf09e33cec1'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:00.232816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] The token cache is being invalidate because project 629311eda0584a5fbb7d8995cf27540c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:00.233223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:00.233901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-81567951-c7be-49c4-8d2c-7cf09e33cec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:00.235494 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1392/2771] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:00 2022] DELETE /identity/v3/projects/629311eda0584a5fbb7d8995cf27540c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:00.240676 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393e34-2b83-403b-abae-9cb6afb258bf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.240965 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393e34-2b83-403b-abae-9cb6afb258bf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.241237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393e34-2b83-403b-abae-9cb6afb258bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.302822 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9b393e34-2b83-403b-abae-9cb6afb258bf None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:00.304004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b393e34-2b83-403b-abae-9cb6afb258bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:00.348326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:00.366287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1380/2772] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:00.374777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8480bd-9090-4561-8632-21f386f40ed2 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.375015 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8480bd-9090-4561-8632-21f386f40ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.375293 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8480bd-9090-4561-8632-21f386f40ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.395924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dd8480bd-9090-4561-8632-21f386f40ed2 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:00.397180 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dd8480bd-9090-4561-8632-21f386f40ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:00.405168 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1393/2773] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:00.409439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b393e34-2b83-403b-abae-9cb6afb258bf None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.419148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b393e34-2b83-403b-abae-9cb6afb258bf None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.424100 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.424793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.424971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.425239 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.425892 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:13:00.426192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.427809 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.439629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1381/2774] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:13:00.444090 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8480bd-9090-4561-8632-21f386f40ed2 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.457179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8480bd-9090-4561-8632-21f386f40ed2 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.462800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.463395 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.463695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.463955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.464625 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.466535 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.505768 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1394/2775] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:00 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:00.510051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.520023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8efc4-2807-443b-9ab1-9805250fdb1d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.580638 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.581242 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.581514 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.581776 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.582405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.584355 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.608781 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1382/2776] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:00 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:00.613149 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.624457 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d37ad-8f17-4c68-b0f4-dc8f50579917 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.630348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.630948 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.631222 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.631490 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.632136 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.633875 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.644320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1395/2777] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:00.648459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.658760 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f54fd-ac73-4b14-93a3-23dfa5839855 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.664344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd0f27-613c-4c02-881f-c5df48467cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.665093 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd0f27-613c-4c02-881f-c5df48467cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.665367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd0f27-613c-4c02-881f-c5df48467cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.665640 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd0f27-613c-4c02-881f-c5df48467cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.666291 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd0f27-613c-4c02-881f-c5df48467cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.667897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd0f27-613c-4c02-881f-c5df48467cc4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.681746 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1383/2778] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:00.683109 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.693270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0f5e-e657-47e9-b6f3-2db91774568a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.698628 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.699315 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.699586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.699861 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] PATH_INFO: `/v3/projects/e6ef3304f1ff453aa1df84c39c5e1010/users/b5f55ee333d2469b9a24061793b528b7/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.716120 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ef3304f1ff453aa1df84c39c5e1010, user_id=b5f55ee333d2469b9a24061793b528b7, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.718742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.737237 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1396/2779] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:00 2022] PUT /identity/v3/projects/e6ef3304f1ff453aa1df84c39c5e1010/users/b5f55ee333d2469b9a24061793b528b7/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:00.748389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387cd3-5fea-4a8d-b9ab-94b9428adef4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.748884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387cd3-5fea-4a8d-b9ab-94b9428adef4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.749018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387cd3-5fea-4a8d-b9ab-94b9428adef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.778281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-54387cd3-5fea-4a8d-b9ab-94b9428adef4 None None] MFA Rules not processed for user `b5f55ee333d2469b9a24061793b528b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:00.779540 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-54387cd3-5fea-4a8d-b9ab-94b9428adef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:00.847114 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1384/2780] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:00.858899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:00.861672 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:00.868558 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:00.938590 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb33689-00bd-479a-8ce7-f57894cda39a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:00.945059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7c025-a50d-4a5d-bd88-494ca5c8ba01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:00.945669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7c025-a50d-4a5d-bd88-494ca5c8ba01 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:00.945951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7c025-a50d-4a5d-bd88-494ca5c8ba01 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:00.946225 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7c025-a50d-4a5d-bd88-494ca5c8ba01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:00.954272 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7c025-a50d-4a5d-bd88-494ca5c8ba01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:00.955851 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7c025-a50d-4a5d-bd88-494ca5c8ba01 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:00.965266 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1397/2781] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:03.669332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387cd3-5fea-4a8d-b9ab-94b9428adef4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:03.714340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:03.733425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387cd3-5fea-4a8d-b9ab-94b9428adef4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:03.738462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:03.739092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:03.739366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:03.739641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] PATH_INFO: `/v3/users/b5f55ee333d2469b9a24061793b528b7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:03.740830 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f55ee333d2469b9a24061793b528b7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:03.742766 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:03.792125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f55ee333d2469b9a24061793b528b7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:03.800232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f55ee333d2469b9a24061793b528b7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:03.800932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f55ee333d2469b9a24061793b528b7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:03.808010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f55ee333d2469b9a24061793b528b7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:03.820841 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f55ee333d2469b9a24061793b528b7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:03.827401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1385/2782] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:13:03 2022] DELETE /identity/v3/users/b5f55ee333d2469b9a24061793b528b7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:03.832278 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:03.832797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:03.832912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:03.852685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:03.853894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:03.914021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1398/2783] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:03.924131 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:03.925948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:03.984310 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa30d1-6f1e-4e64-9e05-a48108f58f46 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:03.988337 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82049fc4-d4b9-4911-8cbc-21b4b4840506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:03.988971 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82049fc4-d4b9-4911-8cbc-21b4b4840506 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:03.989244 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82049fc4-d4b9-4911-8cbc-21b4b4840506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:03.989518 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82049fc4-d4b9-4911-8cbc-21b4b4840506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:03.999026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82049fc4-d4b9-4911-8cbc-21b4b4840506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.000493 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82049fc4-d4b9-4911-8cbc-21b4b4840506 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.009877 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1386/2784] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:04.294302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.295837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:04.308299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc07a33-e779-44c0-983f-e65122489d21 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:04.313310 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:04.313942 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.314221 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.314492 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] PATH_INFO: `/v3/projects/e6ef3304f1ff453aa1df84c39c5e1010` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.316072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ef3304f1ff453aa1df84c39c5e1010)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.317589 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.372658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ef3304f1ff453aa1df84c39c5e1010', 'request_id': 'req-6f2731f8-8f67-43bf-b3c1-89895d927463'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:04.381985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ef3304f1ff453aa1df84c39c5e1010', 'request_id': 'req-6f2731f8-8f67-43bf-b3c1-89895d927463'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:04.387691 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ef3304f1ff453aa1df84c39c5e1010', 'request_id': 'req-6f2731f8-8f67-43bf-b3c1-89895d927463'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:04.398604 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] The token cache is being invalidate because project e6ef3304f1ff453aa1df84c39c5e1010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:04.398891 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:04.399519 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-6f2731f8-8f67-43bf-b3c1-89895d927463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:04.401047 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1399/2785] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:04 2022] DELETE /identity/v3/projects/e6ef3304f1ff453aa1df84c39c5e1010 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:04.406050 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38224-f4c1-4d07-a9a0-310388a8cd43 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.406335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38224-f4c1-4d07-a9a0-310388a8cd43 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.406610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc38224-f4c1-4d07-a9a0-310388a8cd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.425982 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7bc38224-f4c1-4d07-a9a0-310388a8cd43 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:04.427105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7bc38224-f4c1-4d07-a9a0-310388a8cd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:04.486342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1387/2786] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:04.495510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25bfb7-8637-43e8-b960-89ab68cc7063 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.495783 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25bfb7-8637-43e8-b960-89ab68cc7063 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.496026 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25bfb7-8637-43e8-b960-89ab68cc7063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.516520 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-6d25bfb7-8637-43e8-b960-89ab68cc7063 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:04.517750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-6d25bfb7-8637-43e8-b960-89ab68cc7063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:04.526255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1400/2787] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:04.530416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38224-f4c1-4d07-a9a0-310388a8cd43 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.540645 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc38224-f4c1-4d07-a9a0-310388a8cd43 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:04.546704 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:04.547294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.547572 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.547831 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.548520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:13:04.548821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.550728 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.552219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:04.563860 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1388/2788] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:13:04.568474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25bfb7-8637-43e8-b960-89ab68cc7063 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.579342 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25bfb7-8637-43e8-b960-89ab68cc7063 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:04.584495 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:04.585082 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.585358 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.585619 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.586265 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.588153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.592660 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:04.618957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1401/2789] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:04 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:04.625542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.636196 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea95c37-af8f-4aa0-a655-60d17bee9fd9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:04.692323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:04.692996 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.693110 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.693405 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.694031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.695940 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.720759 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1389/2790] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:04 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:04.725370 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.736311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed243b-1fa8-4e31-aaa9-01dd5e2d2a3b admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:04.741060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:04.741656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.741933 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.742195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.742831 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.744415 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.754845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1402/2791] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:04.759013 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.769097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2ea4e-d15d-46f4-9bd8-daa72677d9eb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:04.773858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953b58-80a5-4c30-a06b-f7cecdbef627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:04.774447 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953b58-80a5-4c30-a06b-f7cecdbef627 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.774751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953b58-80a5-4c30-a06b-f7cecdbef627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.775024 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953b58-80a5-4c30-a06b-f7cecdbef627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.775670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953b58-80a5-4c30-a06b-f7cecdbef627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.777362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953b58-80a5-4c30-a06b-f7cecdbef627 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.789209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1390/2792] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:04.793253 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.803154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe680441-f766-4940-b98e-0f13e0b9dae9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:04.809142 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:04.809773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.810057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.810308 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] PATH_INFO: `/v3/projects/e3568338d5734cb1bc955838b2b067d4/users/adaf620deb034a46be1f6cdeb8257876/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.825990 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3568338d5734cb1bc955838b2b067d4, user_id=adaf620deb034a46be1f6cdeb8257876, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:04.828742 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:04.845592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1403/2793] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:04 2022] PUT /identity/v3/projects/e3568338d5734cb1bc955838b2b067d4/users/adaf620deb034a46be1f6cdeb8257876/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:13:04.856015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3b34e-da27-4fe1-a979-5ef2b9f14f21 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:04.856302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3b34e-da27-4fe1-a979-5ef2b9f14f21 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:04.856624 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3b34e-da27-4fe1-a979-5ef2b9f14f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:04.884460 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3ba3b34e-da27-4fe1-a979-5ef2b9f14f21 None None] MFA Rules not processed for user `adaf620deb034a46be1f6cdeb8257876`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:04.885703 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3ba3b34e-da27-4fe1-a979-5ef2b9f14f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:04.953104 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1391/2794] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:04.966722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:04.968725 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:05.043864 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac85ea8-a90a-4428-b051-951061d602ec admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:05.050511 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c855ab-e374-4dc9-8a7c-6aadf328dda3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:05.051106 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c855ab-e374-4dc9-8a7c-6aadf328dda3 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:05.051385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c855ab-e374-4dc9-8a7c-6aadf328dda3 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:05.051663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c855ab-e374-4dc9-8a7c-6aadf328dda3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:05.061556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c855ab-e374-4dc9-8a7c-6aadf328dda3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:05.063927 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c855ab-e374-4dc9-8a7c-6aadf328dda3 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:05.076647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1404/2795] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:05.282630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3b34e-da27-4fe1-a979-5ef2b9f14f21 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:05.350018 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3b34e-da27-4fe1-a979-5ef2b9f14f21 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:05.356069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:05.356726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:05.357000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:05.357267 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] PATH_INFO: `/v3/users/adaf620deb034a46be1f6cdeb8257876` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:05.358490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adaf620deb034a46be1f6cdeb8257876)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:05.360474 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:05.362881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:05.408912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adaf620deb034a46be1f6cdeb8257876'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:05.417586 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adaf620deb034a46be1f6cdeb8257876'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:05.418219 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adaf620deb034a46be1f6cdeb8257876'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:05.425408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adaf620deb034a46be1f6cdeb8257876'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:05.437253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adaf620deb034a46be1f6cdeb8257876'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:05.442988 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1392/2796] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:13:05 2022] DELETE /identity/v3/users/adaf620deb034a46be1f6cdeb8257876 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:05.447641 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:05.447914 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:05.448182 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:05.468656 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:05.469788 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:05.529963 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1405/2797] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:05.539985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:05.541417 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:05.598834 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92668c77-4bbe-4489-8250-2880e773d384 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:05.602868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5afb3c-8b2e-4377-bbd7-eb79479f6b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:05.603464 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5afb3c-8b2e-4377-bbd7-eb79479f6b1b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:05.603736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5afb3c-8b2e-4377-bbd7-eb79479f6b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:05.604031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5afb3c-8b2e-4377-bbd7-eb79479f6b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:05.613305 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5afb3c-8b2e-4377-bbd7-eb79479f6b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:05.614743 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5afb3c-8b2e-4377-bbd7-eb79479f6b1b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:05.624646 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1393/2798] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:05.897774 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:05.899356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:05.911569 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13445d7f-8cd7-4837-bb3d-88b50956b43c None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:05.917709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:05.918372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:05.918658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:05.918930 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] PATH_INFO: `/v3/projects/e3568338d5734cb1bc955838b2b067d4` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:05.920143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3568338d5734cb1bc955838b2b067d4)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:05.922083 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:05.958649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:05.974423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3568338d5734cb1bc955838b2b067d4', 'request_id': 'req-c01d814b-bb80-42ea-a209-4cd0fef0add0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:05.982985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3568338d5734cb1bc955838b2b067d4', 'request_id': 'req-c01d814b-bb80-42ea-a209-4cd0fef0add0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:05.989378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3568338d5734cb1bc955838b2b067d4', 'request_id': 'req-c01d814b-bb80-42ea-a209-4cd0fef0add0'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:06.000866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] The token cache is being invalidate because project e3568338d5734cb1bc955838b2b067d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:06.001161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:06.001797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-c01d814b-bb80-42ea-a209-4cd0fef0add0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:06.003294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1406/2799] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:05 2022] DELETE /identity/v3/projects/e3568338d5734cb1bc955838b2b067d4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:06.008259 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12781e6-5966-4006-9778-42c673250fd9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.008765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12781e6-5966-4006-9778-42c673250fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.008868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12781e6-5966-4006-9778-42c673250fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.028621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a12781e6-5966-4006-9778-42c673250fd9 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:06.029816 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a12781e6-5966-4006-9778-42c673250fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:06.088582 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1394/2800] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:06.096731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337647-401f-48b8-858d-c0610296acaa None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.097016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337647-401f-48b8-858d-c0610296acaa None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.097289 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337647-401f-48b8-858d-c0610296acaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.117652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-63337647-401f-48b8-858d-c0610296acaa None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:06.118846 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-63337647-401f-48b8-858d-c0610296acaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:06.127681 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1407/2801] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:06.132078 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12781e6-5966-4006-9778-42c673250fd9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.142228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12781e6-5966-4006-9778-42c673250fd9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.147694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.148294 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.148629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.148907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.149559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:13:06.149855 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.151833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.164070 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1395/2802] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:13:06.168724 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63337647-401f-48b8-858d-c0610296acaa None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.178162 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63337647-401f-48b8-858d-c0610296acaa None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.183548 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.184143 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.184445 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.184722 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.185360 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.187247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.215164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1408/2803] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:06 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:06.219327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.229575 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39403eb6-4b11-4829-8715-f2e9289866af admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.238068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:06.294097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.294705 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.294983 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.295253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.295914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.297913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.320057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1396/2804] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:06 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:06.326572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.342125 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44e39f-4716-45c5-89e1-fc7fd88a33d6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.348038 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.348825 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.349008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.349257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.350001 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.351642 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.362845 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1409/2805] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:06.366907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.377836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef946c-d067-42d9-8100-a6a57f66e11e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.383127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72203-394c-49fb-aa5b-1da5d3a88f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.383732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72203-394c-49fb-aa5b-1da5d3a88f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.384010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72203-394c-49fb-aa5b-1da5d3a88f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.384279 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72203-394c-49fb-aa5b-1da5d3a88f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.384974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72203-394c-49fb-aa5b-1da5d3a88f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.386606 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72203-394c-49fb-aa5b-1da5d3a88f42 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.397659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1397/2806] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:13:06.402048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.412413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8f45b-1e8d-4eb3-a1ac-eed1b344dac3 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.417547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.418201 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.418469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.418740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] PATH_INFO: `/v3/projects/87450ac2decf4b178af43051a319f701/users/b5d665e0dbaa4bf1b00cce6f0329832d/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.435077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87450ac2decf4b178af43051a319f701, user_id=b5d665e0dbaa4bf1b00cce6f0329832d, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.437785 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.453205 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1410/2807] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:06 2022] PUT /identity/v3/projects/87450ac2decf4b178af43051a319f701/users/b5d665e0dbaa4bf1b00cce6f0329832d/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:06.465183 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712174-b538-4cec-b96c-64997a5fe13f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.465472 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712174-b538-4cec-b96c-64997a5fe13f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.465747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712174-b538-4cec-b96c-64997a5fe13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.494778 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c2712174-b538-4cec-b96c-64997a5fe13f None None] MFA Rules not processed for user `b5d665e0dbaa4bf1b00cce6f0329832d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:06.495959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c2712174-b538-4cec-b96c-64997a5fe13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:06.568442 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1398/2808] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:13:06.573254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.650455 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8ff6-1c89-40d4-9952-1906504fd0e1 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.656356 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.657004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.657285 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.657555 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.658177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.660072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.689161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1411/2809] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:06 2022] POST /identity/v3/projects => generated 403 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:06.694732 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2712174-b538-4cec-b96c-64997a5fe13f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.706088 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2712174-b538-4cec-b96c-64997a5fe13f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.765010 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.765532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.765803 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.766071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.766713 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.768670 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.790123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1399/2810] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:06 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:13:06.794212 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.804526 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067eda6d-c632-4596-bd5c-d555e8839b13 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.809516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.810117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.810394 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.810665 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.811335 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.812966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.823240 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1412/2811] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:06.827410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.837538 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fa914-92f6-4500-96ba-53e3247b892a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.842527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.843161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.843445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.843706 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] PATH_INFO: `/v3/projects/7823caf3ca0549eeaeea7e889b68c32d/users/4075377aa3194347924fb31229313228/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.846893 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:06.861051 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7823caf3ca0549eeaeea7e889b68c32d, user_id=4075377aa3194347924fb31229313228, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.863688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.880620 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1400/2812] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:06 2022] PUT /identity/v3/projects/7823caf3ca0549eeaeea7e889b68c32d/users/4075377aa3194347924fb31229313228/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:06.884718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.894740 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852d09a-2b16-4bd8-a677-45926a27a343 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.920320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:06.953064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c585a9f-21e8-426a-9554-89617315bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.953676 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c585a9f-21e8-426a-9554-89617315bead admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.953954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c585a9f-21e8-426a-9554-89617315bead admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.954229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c585a9f-21e8-426a-9554-89617315bead admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.954897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c585a9f-21e8-426a-9554-89617315bead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.956543 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c585a9f-21e8-426a-9554-89617315bead admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:06.966539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1413/2813] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:06.970878 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:06.981494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b97b80-545c-4322-9b62-1f010f0494c3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:06.986429 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:06.987092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:06.987368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:06.987637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] PATH_INFO: `/v3/projects/7823caf3ca0549eeaeea7e889b68c32d/users/4075377aa3194347924fb31229313228/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:06.989631 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7823caf3ca0549eeaeea7e889b68c32d, user_id=4075377aa3194347924fb31229313228, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:06.992296 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:07.004281 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1401/2814] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:06 2022] PUT /identity/v3/projects/7823caf3ca0549eeaeea7e889b68c32d/users/4075377aa3194347924fb31229313228/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:07.015096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a6f3-e036-4b39-9530-89786601a0d5 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:07.015325 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a6f3-e036-4b39-9530-89786601a0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:07.015599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26a6f3-e036-4b39-9530-89786601a0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:07.044967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ad26a6f3-e036-4b39-9530-89786601a0d5 None None] MFA Rules not processed for user `4075377aa3194347924fb31229313228`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:07.046177 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ad26a6f3-e036-4b39-9530-89786601a0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:07.120563 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1414/2815] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:07.132519 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:07.134512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:07.194780 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eafd0-dd85-4d02-a1ef-ec78c5525d07 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:07.198962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776e56e-a47a-42f7-b252-5db40db3f962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:07.199549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e56e-a47a-42f7-b252-5db40db3f962 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:07.199820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e56e-a47a-42f7-b252-5db40db3f962 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:07.200084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e56e-a47a-42f7-b252-5db40db3f962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:07.209522 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776e56e-a47a-42f7-b252-5db40db3f962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:07.210948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776e56e-a47a-42f7-b252-5db40db3f962 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:07.220245 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1402/2816] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:07 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:07.367391 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26a6f3-e036-4b39-9530-89786601a0d5 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:07.433594 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26a6f3-e036-4b39-9530-89786601a0d5 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:07.438965 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:07.439591 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:07.439868 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:07.440141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] PATH_INFO: `/v3/users/b5d665e0dbaa4bf1b00cce6f0329832d` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:07.441352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d665e0dbaa4bf1b00cce6f0329832d)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:07.443277 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:07.491477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d665e0dbaa4bf1b00cce6f0329832d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:07.495101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:07.502714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d665e0dbaa4bf1b00cce6f0329832d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:07.503697 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d665e0dbaa4bf1b00cce6f0329832d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:07.513057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d665e0dbaa4bf1b00cce6f0329832d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:07.525503 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d665e0dbaa4bf1b00cce6f0329832d'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:07.532053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1415/2817] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:13:07 2022] DELETE /identity/v3/users/b5d665e0dbaa4bf1b00cce6f0329832d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:07.536932 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:07.537212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:07.537477 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:07.557357 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:07.558565 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:07.621107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1403/2818] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:07.633125 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:07.634647 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:07.697554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a367fd-5a68-4964-a9b2-15c87b186d17 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:07.701514 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:07.702133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:07.702408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:07.702682 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:07.711893 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:07.713683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:07.723117 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1416/2819] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:08.005515 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:08.007116 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:08.020002 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9173e3-2a3f-4d02-96c4-60cc5a853605 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:08.025784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:08.027229 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:08.027840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:08.028128 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] PATH_INFO: `/v3/projects/87450ac2decf4b178af43051a319f701` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:08.029364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87450ac2decf4b178af43051a319f701)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:08.031485 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:08.081026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87450ac2decf4b178af43051a319f701', 'request_id': 'req-dda302a1-c713-43bd-a1cc-c722c73779e5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.089611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87450ac2decf4b178af43051a319f701', 'request_id': 'req-dda302a1-c713-43bd-a1cc-c722c73779e5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.096074 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87450ac2decf4b178af43051a319f701', 'request_id': 'req-dda302a1-c713-43bd-a1cc-c722c73779e5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.102690 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:08.108416 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] The token cache is being invalidate because project 87450ac2decf4b178af43051a319f701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:08.108751 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.109546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.111083 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1404/2820] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:08 2022] DELETE /identity/v3/projects/87450ac2decf4b178af43051a319f701 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:08.115471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:08.125810 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7adc0-713b-4e0e-8a82-000f8de5712f service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:08.183475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:08.184102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:08.184377 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:08.184677 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] PATH_INFO: `/v3/users/4075377aa3194347924fb31229313228` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:08.185807 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4075377aa3194347924fb31229313228)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:08.187756 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:08.236574 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4075377aa3194347924fb31229313228'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.245008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4075377aa3194347924fb31229313228'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.245633 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4075377aa3194347924fb31229313228'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.253500 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4075377aa3194347924fb31229313228'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.262072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:08.267464 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dc62bcc3-4863-44c5-9e09-15a1e85a2a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4075377aa3194347924fb31229313228'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.274692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1417/2821] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:13:08 2022] DELETE /identity/v3/users/4075377aa3194347924fb31229313228 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 18:13:08.559979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:08.561461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:08.574836 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda302a1-c713-43bd-a1cc-c722c73779e5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:08.637420 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:08.638212 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:08.638566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:08.638839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] PATH_INFO: `/v3/projects/7823caf3ca0549eeaeea7e889b68c32d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:08.640309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7823caf3ca0549eeaeea7e889b68c32d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:08.642289 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:08.691542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7823caf3ca0549eeaeea7e889b68c32d', 'request_id': 'req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.701566 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7823caf3ca0549eeaeea7e889b68c32d', 'request_id': 'req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.708207 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7823caf3ca0549eeaeea7e889b68c32d', 'request_id': 'req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.718972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] The token cache is being invalidate because project 7823caf3ca0549eeaeea7e889b68c32d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:08.719255 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.719912 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2bde8eb0-dd90-4a0f-99c0-95c4fbd8eb32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:08.721454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1405/2822] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:08 2022] DELETE /identity/v3/projects/7823caf3ca0549eeaeea7e889b68c32d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:08.726375 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35911d5f-1e74-4197-a209-c2782b5b3f2e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:08.726658 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35911d5f-1e74-4197-a209-c2782b5b3f2e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:08.726925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35911d5f-1e74-4197-a209-c2782b5b3f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:08.746931 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-35911d5f-1e74-4197-a209-c2782b5b3f2e None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:08.748113 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-35911d5f-1e74-4197-a209-c2782b5b3f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:08.806152 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1418/2823] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:08.814343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc4f6-76cc-4275-9f30-099d83c4ec03 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:08.814629 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc4f6-76cc-4275-9f30-099d83c4ec03 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:08.814904 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc4f6-76cc-4275-9f30-099d83c4ec03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:08.836380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d2cfc4f6-76cc-4275-9f30-099d83c4ec03 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:08.837622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d2cfc4f6-76cc-4275-9f30-099d83c4ec03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:08.847127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1406/2824] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:08.851554 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35911d5f-1e74-4197-a209-c2782b5b3f2e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:08.861683 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35911d5f-1e74-4197-a209-c2782b5b3f2e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:08.867172 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:08.867782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:08.868072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:08.868347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:08.869048 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:13:08.869341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:08.871344 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:08.883367 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1419/2825] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:13:08.887839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfc4f6-76cc-4275-9f30-099d83c4ec03 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:08.897198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfc4f6-76cc-4275-9f30-099d83c4ec03 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:08.903169 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:08.903800 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:08.904067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:08.904333 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:08.905000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:08.906925 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:08.937559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1407/2826] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:08 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:08.941901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:08.952779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73950321-95b2-4bf3-af85-800a5bc22237 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.019564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.020195 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.020501 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.020760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.021403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.023348 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.046430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1420/2827] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:08 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:13:09.051041 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.061909 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b13ee-da50-4079-9a0e-068f9436871d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.067165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.067765 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.068044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.068311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.069000 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.070621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.080905 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1408/2828] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:09.085127 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.100704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee447b-8c3f-4db8-ab5a-8e8ca03ad49d admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.107797 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced62d29-0e95-4632-996c-d5fd15d871f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.108398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62d29-0e95-4632-996c-d5fd15d871f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.108718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62d29-0e95-4632-996c-d5fd15d871f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.108994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62d29-0e95-4632-996c-d5fd15d871f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.109640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced62d29-0e95-4632-996c-d5fd15d871f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.111238 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced62d29-0e95-4632-996c-d5fd15d871f5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.122690 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1421/2829] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:09.126868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.137789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61304e-f05e-4258-9ec1-16ae1b1f1873 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.142935 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.143511 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.143792 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.144067 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] PATH_INFO: `/v3/projects/e0b27a2684454aa186f0210ba49f8461/users/4b7fe622d97c478a9872ea689a2d137a/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.159122 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b27a2684454aa186f0210ba49f8461, user_id=4b7fe622d97c478a9872ea689a2d137a, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.161832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.177320 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1409/2830] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:09 2022] PUT /identity/v3/projects/e0b27a2684454aa186f0210ba49f8461/users/4b7fe622d97c478a9872ea689a2d137a/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:09.250469 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18fd4c-6c73-4bbc-a502-c7b9d1b226ba None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.251837 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18fd4c-6c73-4bbc-a502-c7b9d1b226ba None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.252298 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18fd4c-6c73-4bbc-a502-c7b9d1b226ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.290306 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-0c18fd4c-6c73-4bbc-a502-c7b9d1b226ba None None] MFA Rules not processed for user `4b7fe622d97c478a9872ea689a2d137a`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:09.291521 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0c18fd4c-6c73-4bbc-a502-c7b9d1b226ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:09.340741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:09.364895 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1422/2831] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:09.370269 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.399898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:09.435898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f479e-2f6d-4330-9d98-d9e6e31bfc59 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.440752 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.441343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.441612 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.441868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.442487 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.444387 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.472082 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1410/2832] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:09 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:09.476273 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18fd4c-6c73-4bbc-a502-c7b9d1b226ba None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.487456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18fd4c-6c73-4bbc-a502-c7b9d1b226ba None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.545275 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.545882 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.546165 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.546434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.547081 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.549060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.571971 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1423/2833] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:09 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 18:13:09.576524 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.587463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132ff91e-77a3-4971-bba5-9dbf532b586b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.593321 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.594204 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.594591 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.594938 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.595581 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.597236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.608076 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1411/2834] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:09.612254 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.624706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9b254-32a9-4c81-a8bf-d754b03e358a admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.630459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50b14-3544-4e14-abb8-c7fec3ea7ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.631070 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50b14-3544-4e14-abb8-c7fec3ea7ece admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.631372 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50b14-3544-4e14-abb8-c7fec3ea7ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.631639 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50b14-3544-4e14-abb8-c7fec3ea7ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.632290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50b14-3544-4e14-abb8-c7fec3ea7ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.634064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50b14-3544-4e14-abb8-c7fec3ea7ece admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.645699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1424/2835] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:09.651697 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.662286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ad497-c769-4a69-a3fc-5b5abc8e2760 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.668964 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.669621 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.669903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.670175 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] PATH_INFO: `/v3/projects/8bada5dde4464fcb9f86abe656dfffbd/users/3ccef538b1d0450a8b68d259005f371c/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.685872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bada5dde4464fcb9f86abe656dfffbd, user_id=3ccef538b1d0450a8b68d259005f371c, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.688674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.702879 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1412/2836] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:09 2022] PUT /identity/v3/projects/8bada5dde4464fcb9f86abe656dfffbd/users/3ccef538b1d0450a8b68d259005f371c/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:09.713836 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d885e69-b3a2-4455-a8dc-cc57c5fe962e None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.714112 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d885e69-b3a2-4455-a8dc-cc57c5fe962e None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.714384 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d885e69-b3a2-4455-a8dc-cc57c5fe962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.742716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-8d885e69-b3a2-4455-a8dc-cc57c5fe962e None None] MFA Rules not processed for user `3ccef538b1d0450a8b68d259005f371c`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:09.743929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d885e69-b3a2-4455-a8dc-cc57c5fe962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:09.812103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1425/2837] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:09.817045 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.880918 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42b41-cf3d-44a8-92d2-4f2dcb1360c2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.886060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.886651 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.886923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.887187 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.887808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.889756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:09.918026 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1413/2838] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:09 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:09.922382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d885e69-b3a2-4455-a8dc-cc57c5fe962e None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:09.933694 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d885e69-b3a2-4455-a8dc-cc57c5fe962e None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:09.985700 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:09.994203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:09.994880 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:09.995158 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:09.995427 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:09.996057 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:09.998365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:10.021016 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1426/2839] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:09 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:10.025452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:10.036571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaad2cb-eda3-4928-a85f-1c9b1efafcaf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:10.042462 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:10.043069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:10.043342 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:10.043610 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:10.044258 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:10.045908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:10.057098 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1414/2840] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:10.063131 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:10.073191 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f328a5-8c2f-4196-b9c2-07faf68a3324 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:10.077955 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:10.078599 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:10.078865 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:10.079133 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] PATH_INFO: `/v3/projects/9d3704c22dbc4b3f80d7e6a960938a9b/users/021b92afd1144030995ecf8dee8fb72c/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:10.093547 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3704c22dbc4b3f80d7e6a960938a9b, user_id=021b92afd1144030995ecf8dee8fb72c, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:10.096164 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:10.110389 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1427/2841] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:10 2022] PUT /identity/v3/projects/9d3704c22dbc4b3f80d7e6a960938a9b/users/021b92afd1144030995ecf8dee8fb72c/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:10.114454 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:10.124948 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fef5a-c51b-4f3d-a742-5481e5827201 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:10.172038 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:10.185503 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ae80a-9716-467f-8a5c-1446db04e77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:10.186102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ae80a-9716-467f-8a5c-1446db04e77b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:10.186386 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ae80a-9716-467f-8a5c-1446db04e77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:10.186667 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ae80a-9716-467f-8a5c-1446db04e77b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:10.187292 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ae80a-9716-467f-8a5c-1446db04e77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:10.188942 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ae80a-9716-467f-8a5c-1446db04e77b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:10.198747 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1415/2842] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:10.203116 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:10.213773 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb8d8-8383-4440-9687-ff5093421ff8 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:10.218755 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:10.219426 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:10.219696 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:10.219967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] PATH_INFO: `/v3/projects/9d3704c22dbc4b3f80d7e6a960938a9b/users/021b92afd1144030995ecf8dee8fb72c/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:10.221915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3704c22dbc4b3f80d7e6a960938a9b, user_id=021b92afd1144030995ecf8dee8fb72c, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:10.224560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:10.234828 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1428/2843] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:10 2022] PUT /identity/v3/projects/9d3704c22dbc4b3f80d7e6a960938a9b/users/021b92afd1144030995ecf8dee8fb72c/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:10.246164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:10.246449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:10.247113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:10.276164 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] MFA Rules not processed for user `021b92afd1144030995ecf8dee8fb72c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:10.277455 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:10.346756 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1416/2844] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:10.357903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:10.360055 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:10.418510 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2590b21-c90b-4eb7-bdfb-795ea5e891d9 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:10.422644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:10.423176 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:10.423452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:10.423728 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:10.437231 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:10.439459 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:10.452033 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1429/2845] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:10.982237 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:10.983974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:10.997833 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00a6cc-befa-4046-bc08-709948886e91 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:11.002453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:11.003054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:11.003332 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:11.003603 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:11.062065 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:11.063512 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:11.072384 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1417/2846] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:15.198560 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:15.207428 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06335257-9824-4a75-b2b2-f2efa13e5f64 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:15.211198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c29690-e535-45e7-b07b-f71e912610ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:15.211789 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c29690-e535-45e7-b07b-f71e912610ee service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:15.212059 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c29690-e535-45e7-b07b-f71e912610ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:15.212322 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c29690-e535-45e7-b07b-f71e912610ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:15.270940 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c29690-e535-45e7-b07b-f71e912610ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:15.272337 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c29690-e535-45e7-b07b-f71e912610ee service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:15.281703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1430/2847] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:17.471628 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:17.534367 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278793d-e7e9-43f3-9220-5c8caf0c1203 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:17.539544 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:17.540157 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:17.540466 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:17.540744 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] PATH_INFO: `/v3/users/4b7fe622d97c478a9872ea689a2d137a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:17.541985 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7fe622d97c478a9872ea689a2d137a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:17.543902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:17.553913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:17.589903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7fe622d97c478a9872ea689a2d137a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:17.599685 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7fe622d97c478a9872ea689a2d137a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:17.600527 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7fe622d97c478a9872ea689a2d137a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:17.607498 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7fe622d97c478a9872ea689a2d137a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:17.620084 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7fe622d97c478a9872ea689a2d137a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:17.627125 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1418/2848] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:13:17 2022] DELETE /identity/v3/users/4b7fe622d97c478a9872ea689a2d137a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:17.634263 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:17.634782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:17.634782 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:17.662246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:17.663456 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:17.711582 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:17.727443 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1431/2849] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:13:17.737569 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:17.739272 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:17.799453 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd2508-3aab-478c-b4a6-9b59af888cd4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:17.803659 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:17.804251 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:17.804549 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:17.804818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:17.814688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:17.816198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:17.826198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1419/2850] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:18.106371 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:18.107818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:18.124929 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dbff6-a60d-4c46-8b11-afc8fd5c54dc None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:18.130702 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:18.131392 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:18.131636 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:18.131892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] PATH_INFO: `/v3/projects/e0b27a2684454aa186f0210ba49f8461` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:18.133134 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b27a2684454aa186f0210ba49f8461)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:18.134992 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:18.185408 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b27a2684454aa186f0210ba49f8461', 'request_id': 'req-0640ccd3-ffb4-4749-a535-546da3cfcb81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.193461 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b27a2684454aa186f0210ba49f8461', 'request_id': 'req-0640ccd3-ffb4-4749-a535-546da3cfcb81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.199049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b27a2684454aa186f0210ba49f8461', 'request_id': 'req-0640ccd3-ffb4-4749-a535-546da3cfcb81'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.209870 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] The token cache is being invalidate because project e0b27a2684454aa186f0210ba49f8461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:18.210154 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.210815 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.212438 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1432/2851] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:18 2022] DELETE /identity/v3/projects/e0b27a2684454aa186f0210ba49f8461 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:18.216716 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:18.226858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320744c5-e1ad-439a-abb8-70a568ea8dff service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:18.290504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:18.291156 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:18.291425 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:18.291688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] PATH_INFO: `/v3/users/3ccef538b1d0450a8b68d259005f371c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:18.292873 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ccef538b1d0450a8b68d259005f371c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:18.294789 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:18.332637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:18.346431 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ccef538b1d0450a8b68d259005f371c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.353866 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ccef538b1d0450a8b68d259005f371c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.354459 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ccef538b1d0450a8b68d259005f371c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.361480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ccef538b1d0450a8b68d259005f371c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.374011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ccef538b1d0450a8b68d259005f371c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.380004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1420/2852] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:13:18 2022] DELETE /identity/v3/users/3ccef538b1d0450a8b68d259005f371c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:18.661036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:18.662539 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:18.676653 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640ccd3-ffb4-4749-a535-546da3cfcb81 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:18.741207 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:18.741850 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:18.742167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:18.742423 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] PATH_INFO: `/v3/projects/8bada5dde4464fcb9f86abe656dfffbd` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:18.743607 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bada5dde4464fcb9f86abe656dfffbd)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:18.745606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:18.795122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bada5dde4464fcb9f86abe656dfffbd', 'request_id': 'req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.802712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bada5dde4464fcb9f86abe656dfffbd', 'request_id': 'req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.808946 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bada5dde4464fcb9f86abe656dfffbd', 'request_id': 'req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.820771 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] The token cache is being invalidate because project 8bada5dde4464fcb9f86abe656dfffbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:18.821060 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.821817 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.823430 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1433/2853] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:18 2022] DELETE /identity/v3/projects/8bada5dde4464fcb9f86abe656dfffbd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:18.827595 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:18.829312 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:18.843057 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e58e1-6251-4256-8c2e-b9a7e3082364 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:18.907913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:18.908494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:18.908775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:18.909046 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] PATH_INFO: `/v3/users/021b92afd1144030995ecf8dee8fb72c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:18.910213 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=021b92afd1144030995ecf8dee8fb72c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:18.912153 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:18.962666 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021b92afd1144030995ecf8dee8fb72c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.969641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021b92afd1144030995ecf8dee8fb72c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.970303 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021b92afd1144030995ecf8dee8fb72c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.977144 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021b92afd1144030995ecf8dee8fb72c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.990771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3823e0bc-16d3-4e28-93d8-8b680a5d4c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021b92afd1144030995ecf8dee8fb72c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:18.997413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1421/2854] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:13:18 2022] DELETE /identity/v3/users/021b92afd1144030995ecf8dee8fb72c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:19.283770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:19.285886 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:19.293915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:19.299860 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddad3aa-0f66-42e0-8547-7448a5a9a4d6 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:19.362334 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:19.362967 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.363241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.363509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] PATH_INFO: `/v3/projects/9d3704c22dbc4b3f80d7e6a960938a9b` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.364703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3704c22dbc4b3f80d7e6a960938a9b)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:19.366586 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:19.413706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3704c22dbc4b3f80d7e6a960938a9b', 'request_id': 'req-1c843b22-ae99-45bf-9e55-acd0785248ad'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:19.424034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3704c22dbc4b3f80d7e6a960938a9b', 'request_id': 'req-1c843b22-ae99-45bf-9e55-acd0785248ad'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:19.429689 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3704c22dbc4b3f80d7e6a960938a9b', 'request_id': 'req-1c843b22-ae99-45bf-9e55-acd0785248ad'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:19.440878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] The token cache is being invalidate because project 9d3704c22dbc4b3f80d7e6a960938a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:13:19.441169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:19.441894 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-1c843b22-ae99-45bf-9e55-acd0785248ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:13:19.443486 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1434/2855] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:13:19 2022] DELETE /identity/v3/projects/9d3704c22dbc4b3f80d7e6a960938a9b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:19.449714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5a58-5eb7-4000-9309-c60d0176b923 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.449997 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5a58-5eb7-4000-9309-c60d0176b923 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.450263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c5a58-5eb7-4000-9309-c60d0176b923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.471260 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1a6c5a58-5eb7-4000-9309-c60d0176b923 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:19.472389 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1a6c5a58-5eb7-4000-9309-c60d0176b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:19.497241 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:19.534430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1422/2856] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:19.543074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d545-0f18-4005-82f5-d14063413dd6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.543365 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d545-0f18-4005-82f5-d14063413dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.543640 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75d545-0f18-4005-82f5-d14063413dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.564559 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-5d75d545-0f18-4005-82f5-d14063413dd6 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:19.565784 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-5d75d545-0f18-4005-82f5-d14063413dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:19.574881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1435/2857] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:19.579232 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c5a58-5eb7-4000-9309-c60d0176b923 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:19.589402 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c5a58-5eb7-4000-9309-c60d0176b923 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:19.595962 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:19.596590 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.596869 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.597137 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.597793 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:13:19.598097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:19.599870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:19.612228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1423/2858] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:13:19.616733 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75d545-0f18-4005-82f5-d14063413dd6 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:19.627096 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75d545-0f18-4005-82f5-d14063413dd6 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:19.633200 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:19.633795 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.634067 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.634341 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.634962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:19.636896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:19.667262 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1436/2859] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:13:19 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:19.671362 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:19.681729 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada29dd-d1b0-413f-a1f5-9942de459357 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:19.739903 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:19.740571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.740846 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.741115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.741761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:19.743709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:19.765715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1424/2860] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:13:19 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:13:19.769909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:19.780580 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c428b3-8ad4-48a7-9026-928c27b676e5 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:19.786075 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:19.786679 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.786960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.787228 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.787866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:19.789517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:19.799561 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1437/2861] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:13:19.803587 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:19.814221 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f5386-17ce-4030-9722-c8a3b9c8680c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:19.819945 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbee13-bf1d-4569-a387-2c1faa5d1c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:19.820577 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbee13-bf1d-4569-a387-2c1faa5d1c15 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.820852 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbee13-bf1d-4569-a387-2c1faa5d1c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.821123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbee13-bf1d-4569-a387-2c1faa5d1c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.821797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbee13-bf1d-4569-a387-2c1faa5d1c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:19.823401 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbee13-bf1d-4569-a387-2c1faa5d1c15 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:19.834228 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1425/2862] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:13:19.838362 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:19.848695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c403472-c4f9-429c-92ef-ea226bcfed56 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:19.853951 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:19.854588 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.854862 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.855129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] PATH_INFO: `/v3/projects/b72151db182e445b9293d7b3b66884ef/users/2661182556d442cea0928f758fe3d57b/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.869731 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72151db182e445b9293d7b3b66884ef, user_id=2661182556d442cea0928f758fe3d57b, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:19.872762 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:19.892723 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1438/2863] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:13:19 2022] PUT /identity/v3/projects/b72151db182e445b9293d7b3b66884ef/users/2661182556d442cea0928f758fe3d57b/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:13:19.899004 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5da41-fbc0-44eb-bfa0-22a37d6c8f8b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:19.899397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5da41-fbc0-44eb-bfa0-22a37d6c8f8b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:19.899775 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5da41-fbc0-44eb-bfa0-22a37d6c8f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:19.921906 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-84e5da41-fbc0-44eb-bfa0-22a37d6c8f8b None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:19.923151 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-84e5da41-fbc0-44eb-bfa0-22a37d6c8f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:19.992452 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1426/2864] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:20.002897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:20.004854 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:20.037036 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:20.073704 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dd8b7-5ea3-4e16-8221-16b287d5e51f admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:20.078803 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:20.079413 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:20.079716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:20.080008 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:20.090387 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:20.091816 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:20.101763 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1439/2865] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:20.748274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2b448-c0ee-46c1-bf8a-0a8c8f9ca808 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:20.748599 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2b448-c0ee-46c1-bf8a-0a8c8f9ca808 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:20.748923 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2b448-c0ee-46c1-bf8a-0a8c8f9ca808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:20.779330 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c7f2b448-c0ee-46c1-bf8a-0a8c8f9ca808 None None] MFA Rules not processed for user `2661182556d442cea0928f758fe3d57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:20.780607 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c7f2b448-c0ee-46c1-bf8a-0a8c8f9ca808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:20.806388 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:20.849919 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1427/2866] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:20.861567 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:20.863115 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:20.923069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0512a1-3f1e-4da5-865e-4a2648af3e5b service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:20.927064 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:20.927675 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:20.927952 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:20.928220 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:20.936800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:20.938304 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:20.947530 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1440/2867] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:22.322773 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:22.323287 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:22.323399 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:22.345532 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] MFA Rules not processed for user `2661182556d442cea0928f758fe3d57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:22.346771 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:22.355227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1428/2868] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:22.601858 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:22.611884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27338107-c3d7-4ad1-a264-f2448177fbd9 service cinder] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:22.616148 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:22.616826 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:22.617129 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:22.617399 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:22.619171 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:22.630347 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:22.631778 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:22.640074 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1441/2869] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:23.994506 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:23.996494 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:24.069960 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d584f-b82f-421d-b22c-9266be55bd80 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:24.075060 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71272b-91eb-4077-a3fa-3118f1912f59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:24.075740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71272b-91eb-4077-a3fa-3118f1912f59 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:24.076011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71272b-91eb-4077-a3fa-3118f1912f59 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:24.076277 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71272b-91eb-4077-a3fa-3118f1912f59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:24.078090 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1e71272b-91eb-4077-a3fa-3118f1912f59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:24.090376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71272b-91eb-4077-a3fa-3118f1912f59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:24.091967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71272b-91eb-4077-a3fa-3118f1912f59 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:24.102480 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1429/2870] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:32.636775 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=45 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:33.867509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:33.869311 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:33.946086 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58d4c1-55dc-4b6f-b82a-108ea5bcae2a service neutron] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:33.952079 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cd340-ab54-4d6b-ba42-eb4fbed79772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:33.952644 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cd340-ab54-4d6b-ba42-eb4fbed79772 service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:33.952922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cd340-ab54-4d6b-ba42-eb4fbed79772 service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:33.953179 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cd340-ab54-4d6b-ba42-eb4fbed79772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:33.955421 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-999cd340-ab54-4d6b-ba42-eb4fbed79772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:33.968407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cd340-ab54-4d6b-ba42-eb4fbed79772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:33.970041 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cd340-ab54-4d6b-ba42-eb4fbed79772 service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:33.981203 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1442/2871] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:13:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:13:34.097870 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=51 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:38.559383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:38.560019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:38.560019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:38.580022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] MFA Rules not processed for user `2661182556d442cea0928f758fe3d57b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:38.581268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:38.589302 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1430/2872] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:13:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 18:13:43.976223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=34 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:48.574165 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:13:57.158714 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8da-9158-4509-96c1-bb463981a27d None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:57.159219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8da-9158-4509-96c1-bb463981a27d None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:57.159219 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8da-9158-4509-96c1-bb463981a27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:57.220403 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-01e1f8da-9158-4509-96c1-bb463981a27d None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:13:57.221643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-01e1f8da-9158-4509-96c1-bb463981a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:57.291901 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1443/2873] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:13:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:13:57.340015 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:13:57.341380 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:13:57.432069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e2569-53e9-4cfd-a964-d8477145e72a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:13:57.437858 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:13:57.438461 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:13:57.438733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:13:57.439001 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:13:57.448102 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:13:57.449718 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:13:57.457783 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1431/2874] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:13:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:14:07.288905 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=44 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:07.454529 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=41 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:13.048866 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce10b3-6925-4cea-9273-cab53fb70c65 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:13.049355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce10b3-6925-4cea-9273-cab53fb70c65 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:13.049472 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce10b3-6925-4cea-9273-cab53fb70c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:13.069417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-13ce10b3-6925-4cea-9273-cab53fb70c65 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:13.070692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-13ce10b3-6925-4cea-9273-cab53fb70c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:13.078398 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1444/2875] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 13 18:14:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:13.090810 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:13.102068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05aca2-535a-4bad-9890-1ba299b5389f service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:13.108634 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:13.109217 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:13.109486 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:13.109753 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:13.118448 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:13.120398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:13.128794 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1432/2876] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:14:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:14:15.050051 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce10b3-6925-4cea-9273-cab53fb70c65 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.067093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce10b3-6925-4cea-9273-cab53fb70c65 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.072777 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.073385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.073661 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.073928 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.075792 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:15.088606 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.090328 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.102229 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1445/2877] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:14:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:14:15.129946 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.141340 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26168486-23e3-4051-9fee-39fdf5d1676c service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.146293 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.146914 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.147188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.147458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] PATH_INFO: `/v3/users/2661182556d442cea0928f758fe3d57b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.148611 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2661182556d442cea0928f758fe3d57b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.150564 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.198344 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2661182556d442cea0928f758fe3d57b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.206730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2661182556d442cea0928f758fe3d57b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.207441 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2661182556d442cea0928f758fe3d57b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.214440 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2661182556d442cea0928f758fe3d57b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.228243 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d2014a5c-c8dc-4384-8a9f-39b638278b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2661182556d442cea0928f758fe3d57b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.234539 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1433/2878] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:14:15 2022] DELETE /identity/v3/users/2661182556d442cea0928f758fe3d57b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:15.437779 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.451962 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7880d51-355e-4e58-a6ee-d0d6736eadc0 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.510896 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.511532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.511814 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.512089 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] PATH_INFO: `/v3/projects/b72151db182e445b9293d7b3b66884ef` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.513257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72151db182e445b9293d7b3b66884ef)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.515187 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.560793 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72151db182e445b9293d7b3b66884ef', 'request_id': 'req-52c0eddb-8442-4740-8450-8965b0cead5e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.568663 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72151db182e445b9293d7b3b66884ef', 'request_id': 'req-52c0eddb-8442-4740-8450-8965b0cead5e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.574417 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72151db182e445b9293d7b3b66884ef', 'request_id': 'req-52c0eddb-8442-4740-8450-8965b0cead5e'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.585877 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] The token cache is being invalidate because project b72151db182e445b9293d7b3b66884ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81429) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:14:15.586167 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.586924 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-52c0eddb-8442-4740-8450-8965b0cead5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:15.588523 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1446/2879] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:14:15 2022] DELETE /identity/v3/projects/b72151db182e445b9293d7b3b66884ef => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:15.593608 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e473f-55af-4a27-8a31-38f270ce14f6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.593895 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e473f-55af-4a27-8a31-38f270ce14f6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.594167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e473f-55af-4a27-8a31-38f270ce14f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.615167 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-623e473f-55af-4a27-8a31-38f270ce14f6 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:15.616435 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-623e473f-55af-4a27-8a31-38f270ce14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:15.677741 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1434/2880] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:15.686160 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42f894-b0a5-4f81-8ea4-9aa5427d9c34 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.686444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42f894-b0a5-4f81-8ea4-9aa5427d9c34 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.686709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42f894-b0a5-4f81-8ea4-9aa5427d9c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.711719 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-1a42f894-b0a5-4f81-8ea4-9aa5427d9c34 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:15.712977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-1a42f894-b0a5-4f81-8ea4-9aa5427d9c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:15.722959 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1447/2881] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:15.727614 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e473f-55af-4a27-8a31-38f270ce14f6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.738449 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e473f-55af-4a27-8a31-38f270ce14f6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.746052 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.746784 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.747048 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.747311 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.747972 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:14:15.748265 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.749913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.766085 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1435/2882] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:14:15.770701 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42f894-b0a5-4f81-8ea4-9aa5427d9c34 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.781902 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42f894-b0a5-4f81-8ea4-9aa5427d9c34 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.788533 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.789140 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.789418 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.789685 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.790320 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.792223 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.822842 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1448/2883] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:14:15 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:14:15.827227 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.837563 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f658089-8474-4846-b91b-621dee5ab33b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.885035 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:15.899517 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.900127 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.900406 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.900693 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.901308 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.903223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.926331 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1436/2884] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:14:15 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:14:15.930752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.941997 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9e340-d78f-452d-82d1-3dad51ac5877 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.947684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.948287 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.948617 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.948890 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.949564 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.951188 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.961699 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1449/2885] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:14:15.965674 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:15.975975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bedc5-1091-4be9-9a9a-eb4088371f7c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:15.981445 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6513c0b-7db4-4792-b47e-f18b153d8657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:15.982107 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6513c0b-7db4-4792-b47e-f18b153d8657 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:15.982439 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6513c0b-7db4-4792-b47e-f18b153d8657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:15.982736 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6513c0b-7db4-4792-b47e-f18b153d8657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:15.983398 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6513c0b-7db4-4792-b47e-f18b153d8657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:15.985179 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6513c0b-7db4-4792-b47e-f18b153d8657 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:15.996902 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1437/2886] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:14:16.001477 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:16.012043 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386f4bd-ae96-4cce-ae2b-db2e4ae17dc7 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:16.017262 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:16.017917 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:16.018197 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:16.018471 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] PATH_INFO: `/v3/projects/655c0fd331c343fa8b451c98cc368f5d/users/dd59ce3705d94670b825d5fdb03a9472/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:16.034973 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=655c0fd331c343fa8b451c98cc368f5d, user_id=dd59ce3705d94670b825d5fdb03a9472, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:16.039452 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:16.057141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1450/2887] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:14:16 2022] PUT /identity/v3/projects/655c0fd331c343fa8b451c98cc368f5d/users/dd59ce3705d94670b825d5fdb03a9472/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:16.068821 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada7ce-be81-4a79-96c5-19fbbd73915b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:16.069225 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada7ce-be81-4a79-96c5-19fbbd73915b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:16.069622 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada7ce-be81-4a79-96c5-19fbbd73915b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:16.108525 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3fada7ce-be81-4a79-96c5-19fbbd73915b None None] MFA Rules not processed for user `dd59ce3705d94670b825d5fdb03a9472`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:16.109761 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3fada7ce-be81-4a79-96c5-19fbbd73915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:16.193605 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1438/2888] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:16.204450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:16.206434 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:16.216939 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:16.266462 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd5f5d-bbad-4e82-bd17-0e18f764f9ae admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:16.270364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec712ba-069b-473e-8647-793861ad0f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9c5205f9344e8c8fcc85c80d6f504b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:16.270977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec712ba-069b-473e-8647-793861ad0f56 service cinder] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:16.271244 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec712ba-069b-473e-8647-793861ad0f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:16.271509 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec712ba-069b-473e-8647-793861ad0f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:16.281139 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec712ba-069b-473e-8647-793861ad0f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:16.282695 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec712ba-069b-473e-8647-793861ad0f56 service cinder] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:16.291957 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1451/2889] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:14:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:14:17.688898 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:17.689490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:17.689490 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:17.709475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] MFA Rules not processed for user `dd59ce3705d94670b825d5fdb03a9472`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:17.710668 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:17.718054 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1439/2890] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:26.287760 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=23 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:27.703709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:31.851800 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1e20-1337-49ac-8f52-a0edaf6a2cd4 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:31.852282 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1e20-1337-49ac-8f52-a0edaf6a2cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:31.852409 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d1e20-1337-49ac-8f52-a0edaf6a2cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:31.874463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-426d1e20-1337-49ac-8f52-a0edaf6a2cd4 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:31.875669 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-426d1e20-1337-49ac-8f52-a0edaf6a2cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:31.930326 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1452/2891] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:14:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:31.945236 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:31.947071 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:32.026641 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ff269-3f95-4143-8eca-569f670d5fbf None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:32.035092 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:32.035913 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:32.036297 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:32.036709 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:32.046653 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:32.048233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:32.056263 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1440/2892] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:14:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:14:32.594184 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0be08-a49a-4b58-a340-911b079f2fc0 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:32.594475 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0be08-a49a-4b58-a340-911b079f2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:32.594750 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0be08-a49a-4b58-a340-911b079f2fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:32.615107 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-09e0be08-a49a-4b58-a340-911b079f2fc0 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:32.616366 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-09e0be08-a49a-4b58-a340-911b079f2fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:32.625566 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1453/2893] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 13 18:14:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:32.641304 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:32.652458 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfb643-1966-4dc3-aed1-06045b493d8d service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:32.658907 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:32.659495 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:32.659763 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:32.660023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:32.668364 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:32.669975 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:32.678072 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1441/2894] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:14:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 18:14:32.924578 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ad29-a5ce-49cf-b34e-179322b18c28 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:32.925103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ad29-a5ce-49cf-b34e-179322b18c28 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:32.925103 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ad29-a5ce-49cf-b34e-179322b18c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:32.944897 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-eb28ad29-a5ce-49cf-b34e-179322b18c28 None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:32.946084 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-eb28ad29-a5ce-49cf-b34e-179322b18c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:32.955279 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1454/2895] 10.222.0.43 () {58 vars in 1009 bytes} [Tue Sep 13 18:14:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:32.967274 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:32.978520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636aba-0b24-4a12-be4b-0bfc085ad49f service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:32.984414 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:32.985023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:32.985299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:32.985593 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:32.994044 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:32.996011 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:33.003956 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1442/2896] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:14:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:14:33.328071 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1f51-51bb-46ca-bcd9-95d522a9a1ca None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:33.328354 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1f51-51bb-46ca-bcd9-95d522a9a1ca None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:33.328655 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a1f51-51bb-46ca-bcd9-95d522a9a1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:33.349909 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-ec7a1f51-51bb-46ca-bcd9-95d522a9a1ca None None] MFA Rules not processed for user `1e6a922a2aea4d3ea1feadbd25cb32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:33.351101 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-ec7a1f51-51bb-46ca-bcd9-95d522a9a1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:33.358754 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1455/2897] 10.222.0.43 () {58 vars in 967 bytes} [Tue Sep 13 18:14:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:33.371590 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:33.382899 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c894a-07e9-4668-bede-a1bf2c9868a1 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:33.389327 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af3e63-3cd3-4bea-aecf-d9304406ab5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ec763f1afac469196a5e897587813b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:33.389921 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af3e63-3cd3-4bea-aecf-d9304406ab5d service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:33.390197 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af3e63-3cd3-4bea-aecf-d9304406ab5d service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:33.390463 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af3e63-3cd3-4bea-aecf-d9304406ab5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:33.398959 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af3e63-3cd3-4bea-aecf-d9304406ab5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:33.400546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af3e63-3cd3-4bea-aecf-d9304406ab5d service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:33.409443 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1443/2898] 10.222.0.43 () {60 vars in 1384 bytes} [Tue Sep 13 18:14:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 18:14:35.930085 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1f51-51bb-46ca-bcd9-95d522a9a1ca None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.001035 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a1f51-51bb-46ca-bcd9-95d522a9a1ca None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.006141 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.006752 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.007028 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.007364 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] PATH_INFO: `/v3/users/dd59ce3705d94670b825d5fdb03a9472` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.008592 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd59ce3705d94670b825d5fdb03a9472)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.010516 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.057859 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd59ce3705d94670b825d5fdb03a9472'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.066246 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd59ce3705d94670b825d5fdb03a9472'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.066843 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd59ce3705d94670b825d5fdb03a9472'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.073878 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd59ce3705d94670b825d5fdb03a9472'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.087256 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd59ce3705d94670b825d5fdb03a9472'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.093230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1456/2899] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:14:35 2022] DELETE /identity/v3/users/dd59ce3705d94670b825d5fdb03a9472 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:36.097884 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.098383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.098383 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.118559 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:36.120118 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:36.181226 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1444/2900] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:36.191257 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.192744 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:36.200021 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:36.252382 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266eb8-43b3-4b6a-be4d-ce54d843e2af admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.256331 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4804fa-0959-4351-8782-a7c48bacb2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.256969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4804fa-0959-4351-8782-a7c48bacb2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.257245 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4804fa-0959-4351-8782-a7c48bacb2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.257517 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4804fa-0959-4351-8782-a7c48bacb2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.266938 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4804fa-0959-4351-8782-a7c48bacb2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.268402 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4804fa-0959-4351-8782-a7c48bacb2c7 service neutron] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.277985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1457/2901] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:14:36.450352 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.451872 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:36.463715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd1bda-0236-401b-9ef3-5c6b931da225 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.468397 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.469825 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.470113 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.470504 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] PATH_INFO: `/v3/projects/655c0fd331c343fa8b451c98cc368f5d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.472115 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=655c0fd331c343fa8b451c98cc368f5d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.474119 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.526714 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:36.529927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '655c0fd331c343fa8b451c98cc368f5d', 'request_id': 'req-304bc105-30b8-4d10-b8a4-2a4d473b4c31'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.538022 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '655c0fd331c343fa8b451c98cc368f5d', 'request_id': 'req-304bc105-30b8-4d10-b8a4-2a4d473b4c31'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.544979 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '655c0fd331c343fa8b451c98cc368f5d', 'request_id': 'req-304bc105-30b8-4d10-b8a4-2a4d473b4c31'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.555408 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] The token cache is being invalidate because project 655c0fd331c343fa8b451c98cc368f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:14:36.555694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.556326 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-304bc105-30b8-4d10-b8a4-2a4d473b4c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:14:36.557859 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1445/2902] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:14:36 2022] DELETE /identity/v3/projects/655c0fd331c343fa8b451c98cc368f5d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:36.563123 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3486fa-747d-4ed3-adaf-e34de5d08de8 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.563474 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3486fa-747d-4ed3-adaf-e34de5d08de8 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.563749 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3486fa-747d-4ed3-adaf-e34de5d08de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.583912 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-bf3486fa-747d-4ed3-adaf-e34de5d08de8 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:36.585146 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bf3486fa-747d-4ed3-adaf-e34de5d08de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:36.652741 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1458/2903] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:36.660818 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d344b3d-1d09-4dc9-a064-573b43fc668c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.661105 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d344b3d-1d09-4dc9-a064-573b43fc668c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.661376 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d344b3d-1d09-4dc9-a064-573b43fc668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.682176 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9d344b3d-1d09-4dc9-a064-573b43fc668c None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:36.683371 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9d344b3d-1d09-4dc9-a064-573b43fc668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:36.692091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1446/2904] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:36.696649 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3486fa-747d-4ed3-adaf-e34de5d08de8 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.707091 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3486fa-747d-4ed3-adaf-e34de5d08de8 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.713302 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.713904 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.714169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.714444 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.715077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81429) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 18:14:36.715404 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.717049 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.729401 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1459/2905] 10.222.0.43 () {58 vars in 1153 bytes} [Tue Sep 13 18:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 18:14:36.733772 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d344b3d-1d09-4dc9-a064-573b43fc668c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.743286 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d344b3d-1d09-4dc9-a064-573b43fc668c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.748547 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.749148 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.749430 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.749715 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.750343 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.752280 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.782261 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1447/2906] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:14:36 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:14:36.787077 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.797969 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359783f-b3ca-4237-a654-bc69e2630cd4 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.854770 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.855352 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.855637 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.855903 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.856556 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.858482 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.882022 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1460/2907] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:14:36 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:14:36.886353 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.896927 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa0a44-b784-48f0-8708-5f99e238e09f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.902021 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.902630 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.902908 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.903161 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.903815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.905469 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.915497 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1448/2908] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:14:36.919247 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.929230 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592926d-5a4a-42c6-b562-584ffbf294fd admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.934355 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07caf4a-7857-46de-a9c4-18ab5e2dd56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.934966 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07caf4a-7857-46de-a9c4-18ab5e2dd56a admin admin] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.935196 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07caf4a-7857-46de-a9c4-18ab5e2dd56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.935463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07caf4a-7857-46de-a9c4-18ab5e2dd56a admin admin] PATH_INFO: `/v3/roles` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.936093 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07caf4a-7857-46de-a9c4-18ab5e2dd56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.937703 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07caf4a-7857-46de-a9c4-18ab5e2dd56a admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:36.948497 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1461/2909] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:14:36.952657 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:36.963123 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658adcf1-3cd1-4fba-b5ac-e9b7efb9b4a2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:36.968911 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:36.969542 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:36.969820 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:36.970086 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] PATH_INFO: `/v3/projects/0fcaaf852cd6415c96656b52df5ccef8/users/0a18300461ee48e8900f9d182fdc72c7/roles/5c592b5b170d48b0825640b8283425de` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:36.984970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fcaaf852cd6415c96656b52df5ccef8, user_id=0a18300461ee48e8900f9d182fdc72c7, role_id=5c592b5b170d48b0825640b8283425de)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:36.987616 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:37.005993 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1449/2910] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:14:36 2022] PUT /identity/v3/projects/0fcaaf852cd6415c96656b52df5ccef8/users/0a18300461ee48e8900f9d182fdc72c7/roles/5c592b5b170d48b0825640b8283425de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:37.011922 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213250-9150-424b-b771-286b21db79c6 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:37.012351 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213250-9150-424b-b771-286b21db79c6 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:37.012832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213250-9150-424b-b771-286b21db79c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:37.036004 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-24213250-9150-424b-b771-286b21db79c6 None None] MFA Rules not processed for user `bb561c12b9c44bd29f9bae620dc33608`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:37.037204 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-24213250-9150-424b-b771-286b21db79c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:37.046294 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:37.101821 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1462/2911] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:37.112270 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:37.115158 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:37.176862 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5219af-c01e-47d0-bd7e-36c8aab62141 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:37.181184 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b10367002d4f64a82b3e9eca00ca06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:37.181797 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:37.182068 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:37.182335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:37.192808 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:37.195091 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:37.210840 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1450/2912] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:14:41.188532 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2af4a-bc08-4765-8786-69a92cbbb5a7 None None] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:41.189102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2af4a-bc08-4765-8786-69a92cbbb5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:41.189102 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2af4a-bc08-4765-8786-69a92cbbb5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:41.223209 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.auth.core [None req-78f2af4a-bc08-4765-8786-69a92cbbb5a7 None None] MFA Rules not processed for user `0a18300461ee48e8900f9d182fdc72c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81429) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:41.224813 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-78f2af4a-bc08-4765-8786-69a92cbbb5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:41.299986 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1463/2913] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:41.301306 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:41.311868 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31a1d9-224b-4544-8cc3-aa695c018cdc service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:41.317064 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:41.317867 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:41.318147 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:41.318413 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:41.319097 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:41.321257 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:41.352481 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1451/2914] 10.222.0.43 () {60 vars in 1153 bytes} [Tue Sep 13 18:14:41 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:14:41.355545 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2af4a-bc08-4765-8786-69a92cbbb5a7 None None] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:41.366412 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2af4a-bc08-4765-8786-69a92cbbb5a7 None None] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:41.426620 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:41.427153 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] REQUEST_METHOD: `POST` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:41.427439 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:41.427706 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] PATH_INFO: `/v3/users` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:41.428385 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:41.430407 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:41.454128 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1464/2915] 10.222.0.43 () {60 vars in 1144 bytes} [Tue Sep 13 18:14:41 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 18:14:41.458410 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:41.469323 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97ce3-6651-4b14-8114-b5b5b7217cae admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:41.474600 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:41.475198 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:41.475482 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:41.475757 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:41.476598 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:41.478526 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:41.490368 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1452/2916] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 18:14:41.495692 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:41.506010 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126d54d-b770-4866-b078-ae42b7116ffa admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:41.511069 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:41.511718 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:41.511996 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:41.512290 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] PATH_INFO: `/v3/projects/67429900f8ed4d4796714e341668f362/users/e6ec1800d8f44a86bb3d5f28bd547b63/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:41.528802 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67429900f8ed4d4796714e341668f362, user_id=e6ec1800d8f44a86bb3d5f28bd547b63, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:41.531481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:41.546361 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1465/2917] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:14:41 2022] PUT /identity/v3/projects/67429900f8ed4d4796714e341668f362/users/e6ec1800d8f44a86bb3d5f28bd547b63/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:41.550475 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:41.564754 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290c812-f77b-4276-b22f-fb04ae8bdae5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:41.615724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:41.640688 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de307d0b-1e6a-4bfb-94a0-ed0f36ed9940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:41.641468 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de307d0b-1e6a-4bfb-94a0-ed0f36ed9940 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:41.641850 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de307d0b-1e6a-4bfb-94a0-ed0f36ed9940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:41.642233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de307d0b-1e6a-4bfb-94a0-ed0f36ed9940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:41.643132 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de307d0b-1e6a-4bfb-94a0-ed0f36ed9940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:41.645724 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de307d0b-1e6a-4bfb-94a0-ed0f36ed9940 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:41.656953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1453/2918] 10.222.0.43 () {58 vars in 1122 bytes} [Tue Sep 13 18:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 18:14:41.661350 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:41.671925 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89b078-0183-45fe-bee0-649b7763ae62 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:41.677118 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:41.677713 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:41.677994 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:41.678270 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] PATH_INFO: `/v3/projects/67429900f8ed4d4796714e341668f362/users/e6ec1800d8f44a86bb3d5f28bd547b63/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:41.680329 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67429900f8ed4d4796714e341668f362, user_id=e6ec1800d8f44a86bb3d5f28bd547b63, role_id=9fe95b74b86c4f2fa8a7b5712b7b3d0a)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:41.683072 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:41.686481 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:41.695108 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1466/2919] 10.222.0.43 () {60 vars in 1483 bytes} [Tue Sep 13 18:14:41 2022] PUT /identity/v3/projects/67429900f8ed4d4796714e341668f362/users/e6ec1800d8f44a86bb3d5f28bd547b63/roles/9fe95b74b86c4f2fa8a7b5712b7b3d0a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:14:46.114209 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed6ef9-a56e-4111-ac46-2284daff8c0d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:46.114740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed6ef9-a56e-4111-ac46-2284daff8c0d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:46.114740 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed6ef9-a56e-4111-ac46-2284daff8c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:46.145295 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bfed6ef9-a56e-4111-ac46-2284daff8c0d None None] MFA Rules not processed for user `e6ec1800d8f44a86bb3d5f28bd547b63`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:14:46.146635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bfed6ef9-a56e-4111-ac46-2284daff8c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:46.232815 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1454/2920] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:14:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:14:46.251255 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:14:46.253441 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:14:46.330241 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724a5b-b144-41a6-925e-064c50c92b19 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:14:46.335709 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90f40fe8f20f4a5bbd201debda6a97a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:14:46.336332 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:14:46.336643 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:14:46.336915 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:14:46.406378 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:14:46.408571 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:14:46.422847 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1467/2921] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:14:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:14:56.229832 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=45 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:14:56.417299 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=58 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:15:01.859839 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ba94c-a1d4-4afe-93c7-57db2e19b7d5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:15:01.860366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ba94c-a1d4-4afe-93c7-57db2e19b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:15:01.860366 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ba94c-a1d4-4afe-93c7-57db2e19b7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:15:01.885479 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d91ba94c-a1d4-4afe-93c7-57db2e19b7d5 None None] MFA Rules not processed for user `0a18300461ee48e8900f9d182fdc72c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 18:15:01.886694 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d91ba94c-a1d4-4afe-93c7-57db2e19b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:15:01.894974 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1455/2922] 10.222.0.43 () {58 vars in 958 bytes} [Tue Sep 13 18:15:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 18:15:11.878623 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:15:58.298977 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:15:58.300716 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:15:58.373884 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3307ac-999f-49c1-8742-aed96a8f5f0d service nova] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:15:58.379960 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:15:58.380616 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:15:58.380899 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:15:58.381198 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:15:58.383053 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:15:58.395383 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:15:58.397034 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:15:58.407292 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1468/2923] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:15:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:16:06.513897 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ba94c-a1d4-4afe-93c7-57db2e19b7d5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:16:06.525786 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ba94c-a1d4-4afe-93c7-57db2e19b7d5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:16:06.531031 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:16:06.532023 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:16:06.532546 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:16:06.533029 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:16:06.534967 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:16:06.547056 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:16:06.549019 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:16:06.560268 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1456/2924] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:16:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:16:08.402820 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=32 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:16:09.846460 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:16:09.910450 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d34695-9444-4b82-859d-adb251a569a4 service placement] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:16:09.915193 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:16:09.915824 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:16:09.916105 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:16:09.916380 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] PATH_INFO: `/v3/users/0a18300461ee48e8900f9d182fdc72c7` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:16:09.917680 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a18300461ee48e8900f9d182fdc72c7)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:16:09.919879 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:16:09.967499 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a18300461ee48e8900f9d182fdc72c7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:09.976181 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a18300461ee48e8900f9d182fdc72c7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:09.976985 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a18300461ee48e8900f9d182fdc72c7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:09.985122 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a18300461ee48e8900f9d182fdc72c7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.001572 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a18300461ee48e8900f9d182fdc72c7'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.007954 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1469/2925] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:16:09 2022] DELETE /identity/v3/users/0a18300461ee48e8900f9d182fdc72c7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:16:10.199689 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:16:10.212730 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06446a-6d8c-451c-b046-8a8ea474c78d service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:16:10.270053 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:16:10.270687 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:16:10.270961 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:16:10.271223 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] PATH_INFO: `/v3/projects/0fcaaf852cd6415c96656b52df5ccef8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:16:10.272523 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fcaaf852cd6415c96656b52df5ccef8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:16:10.274520 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:16:10.321970 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fcaaf852cd6415c96656b52df5ccef8', 'request_id': 'req-72a88089-0d9d-4a6b-ba22-91b864d60dba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.329253 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fcaaf852cd6415c96656b52df5ccef8', 'request_id': 'req-72a88089-0d9d-4a6b-ba22-91b864d60dba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.335637 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fcaaf852cd6415c96656b52df5ccef8', 'request_id': 'req-72a88089-0d9d-4a6b-ba22-91b864d60dba'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.345826 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] The token cache is being invalidate because project 0fcaaf852cd6415c96656b52df5ccef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:16:10.346111 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.346733 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.348188 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1457/2926] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:16:10 2022] DELETE /identity/v3/projects/0fcaaf852cd6415c96656b52df5ccef8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:16:10.352208 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:16:10.353652 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:16:10.367100 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88ee0-7de7-4a44-b1b8-117eadcb0476 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:16:10.441712 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:16:10.442393 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:16:10.442671 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:16:10.442944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] PATH_INFO: `/v3/users/e6ec1800d8f44a86bb3d5f28bd547b63` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:16:10.444161 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ec1800d8f44a86bb3d5f28bd547b63)` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:16:10.446248 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:16:10.472249 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:16:10.509684 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ec1800d8f44a86bb3d5f28bd547b63'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.518169 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ec1800d8f44a86bb3d5f28bd547b63'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.518892 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ec1800d8f44a86bb3d5f28bd547b63'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.526098 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ec1800d8f44a86bb3d5f28bd547b63'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.538801 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.notifications [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ec1800d8f44a86bb3d5f28bd547b63'} {{(pid=81429) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.544832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1470/2927] 10.222.0.43 () {58 vars in 1224 bytes} [Tue Sep 13 18:16:10 2022] DELETE /identity/v3/users/e6ec1800d8f44a86bb3d5f28bd547b63 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:16:10.727571 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:16:10.729089 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:16:10.744881 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a88089-0d9d-4a6b-ba22-91b864d60dba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:16:10.814309 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb561c12b9c44bd29f9bae620dc33608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c326683f56fe42d09b126fc368517cb3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:16:10.814953 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:16:10.815233 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:16:10.815505 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] PATH_INFO: `/v3/projects/67429900f8ed4d4796714e341668f362` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:16:10.816696 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67429900f8ed4d4796714e341668f362)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:16:10.818635 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:16:10.839335 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:16:10.866299 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67429900f8ed4d4796714e341668f362', 'request_id': 'req-e082e31e-d844-4cf6-86f6-3c40b435bde1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.876200 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67429900f8ed4d4796714e341668f362', 'request_id': 'req-e082e31e-d844-4cf6-86f6-3c40b435bde1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.882880 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67429900f8ed4d4796714e341668f362', 'request_id': 'req-e082e31e-d844-4cf6-86f6-3c40b435bde1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.893726 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] The token cache is being invalidate because project 67429900f8ed4d4796714e341668f362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 18:16:10.894069 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.894627 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e082e31e-d844-4cf6-86f6-3c40b435bde1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 18:16:10.896192 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1458/2928] 10.222.0.43 () {58 vars in 1233 bytes} [Tue Sep 13 18:16:10 2022] DELETE /identity/v3/projects/67429900f8ed4d4796714e341668f362 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 18:16:20.541463 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:16:20.891108 dv-cinder-852009-8-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 18:21:04.295504 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Authenticating user token {{(pid=81429) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 18:21:04.297190 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:21:04.449818 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf565a1b-544b-40a1-a74b-d559f393f366 admin admin] Validating token access rules against request {{(pid=81429) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 18:21:04.511632 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db052e-47e1-4138-b56b-55163a4dc5ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6ce465c1ee84703b7b44d33a91d7017', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2b736c723c894226850e560a2dd28985', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81429) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 18:21:04.512239 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db052e-47e1-4138-b56b-55163a4dc5ce service placement] REQUEST_METHOD: `GET` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 18:21:04.512544 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db052e-47e1-4138-b56b-55163a4dc5ce service placement] SCRIPT_NAME: `/identity` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 18:21:04.512822 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db052e-47e1-4138-b56b-55163a4dc5ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81429) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 18:21:04.514832 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.fernet_utils [None req-f6db052e-47e1-4138-b56b-55163a4dc5ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81429) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 18:21:04.526381 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db052e-47e1-4138-b56b-55163a4dc5ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 18:21:04.527944 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db052e-47e1-4138-b56b-55163a4dc5ce service placement] RBAC: Authorization granted {{(pid=81429) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 18:21:04.537881 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: [pid: 81429|app: 0|req: 1471/2929] 10.222.0.43 () {60 vars in 1365 bytes} [Tue Sep 13 18:21:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 18:21:14.533550 dv-cinder-852009-8-iscsi devstack@keystone.service[81429]: DEBUG dbcounter [-] [81429] Writing DB stats keystone:SELECT=89 {{(pid=81429) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}